From 4816a8c1bc8959a20c3d2aa06b4e02083b4d39e8 Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Thu, 13 Aug 2026 15:33:53 -0700 Subject: [PATCH 1/4] =?UTF-8?q?fix(gem):=20bundler=20vendored+hosted=20swe?= =?UTF-8?q?ep=20=E2=80=94=2013=20confirmed=20bugs=20fixed,=20tests=20harde?= =?UTF-8?q?ned?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit ULTRACODE review + full test matrix over the gem/bundler ecosystem (vendored and hosted modes, every configuration). 24 findings survived adversarial verification; the 13 well-scoped ones are fixed here, each pinned by a test that fails without its fix. Hosted redirect (patch/redirect/mod.rs, gem section): - splice the Gemfile edit by regex byte range: a commented-out duplicate of the gem line no longer gets rewritten instead of the live line - grant-agnostic idempotency guard: re-running scan --mode hosted with a rotated grant token refreshes the source URL in place instead of nesting a second source block (new edit kind redirect_gemfile_source_url) - fail closed on gem-level git:/github:/path:/source: options (they override the enclosing source block, making the redirect an attested no-op); warn redirect_gem_source_option and skip the dep - fail closed on platform-suffixed CHECKSUMS siblings (redirect_gem_platform_unsupported) instead of inserting a duplicate bare-coordinate pin - recognize paren/tab/multi-space gem declarations; gate the append-branch on the gem being genuinely undeclared (no more duplicate declarations bundler rejects) - never pin the lock CHECKSUMS when the Gemfile source redirect did not land (mixed state guaranteed a checksum failure) - warn that a redirected pair breaks frozen/deployment installs - record the upstream sha256 line as original on the CHECKSUMS edit so a future revert can restore it (golden fixture updated) Vendored backend (vendor/gem.rs): - insert new PATH sections at bundler's sorted position (identifier order, verified against real bundler 4.0.15 bundle lock) — two or more vendored gems no longer churn the committed lock - fail closed on platform-suffixed GEM-specs siblings on no-CHECKSUMS locks (mirrors the existing CHECKSUMS guard) - re-vendor on a patch UPDATE (new uuid, same purl): recognize our own path: wiring and rewire in place instead of refusing with gemfile_declaration_not_editable — the documented automatic re-vendor contract now actually works for gem Auto-fetch (vendor/registry_fetch.rs): - stage fetched gems into the canonical - leaf instead of a dir literally named "gem", which vendor_gem refused as platform_gem_unsupported — lockfile auto-fetch for gems was dead Crawler (crawlers/ruby_crawler.rs): - parse_dir_name_version prefers the last dotted-version boundary, so http-2-1.0.1 parses as (http-2, 1.0.1) instead of the ghost (http, 2) - vendor/bundle discovery enumerates engine dirs (jruby, truffleruby) instead of hardcoding ruby/ Scan/get plumbing: - run_nested_apply now threads --ecosystems: scan --ecosystems gem --sync no longer applies (or mutates) other ecosystems' patches - scan --vendor --dry-run --vex no longer writes the VEX file nor exits 1 on not-yet-vendored state Test hardening: - e2e_gem lifecycle harness: BUNDLE_PATH replaces bundle install --path (removed in bundler 3+; all 3 lifecycle tests green under 4.0.15) - e2e_hosted_production gem leg now asserts the CHECKSUMS digest CHANGED after redirect (client-verifiable today) and, in the success arm, verifies installed content against the published afterHashes — an inert gem patch can no longer stay green (the npm minimist blindspot) - docker_e2e_gem serves the true git-blob beforeHash so the chain exercises the default non-forced apply path, not just --force - setup_matrix_gem module doc: the with-setup Docker cases ARE still a baseline gap (bootstrap deadlock: plugin registration evaluates plugins.rb before any gems land; exit-semantics twin), doc corrected Verified: core 2460/0, cli lib 350/0, clippy+fmt clean; e2e_gem 11/11 (incl. live lifecycle under bundler 4.0.15), e2e_vendor_gem_build 6/6 (incl. real-bundler capstone), docker_e2e_gem 2/2, docker_e2e_vendor_gem 1/1, hosted production gem leg green (redirect verified; install still blocked by the known depscan#23630 compact-index 404 — server-side). Co-Authored-By: Claude Fable 5 --- crates/socket-patch-cli/src/commands/get.rs | 24 +- .../socket-patch-cli/src/commands/scan/mod.rs | 1 + .../src/commands/scan/vendor_flow.rs | 17 +- .../socket-patch-cli/tests/docker_e2e_gem.rs | 112 ++- crates/socket-patch-cli/tests/e2e_gem.rs | 11 +- .../tests/e2e_hosted_production.rs | 169 ++++ .../tests/in_process_get_manifest_path.rs | 181 +++++ .../tests/in_process_get_update_count.rs | 1 + .../socket-patch-cli/tests/in_process_scan.rs | 107 +++ .../tests/setup_matrix_gem.rs | 16 +- .../src/crawlers/ruby_crawler.rs | 162 +++- .../src/patch/redirect/mod.rs | 563 +++++++++++++- crates/socket-patch-core/src/vendor/gem.rs | 735 +++++++++++++++++- .../src/vendor/registry_fetch.rs | 45 +- .../tests/crawler_ruby_e2e.rs | 41 + .../gem/bundler/basic/expected-edits.json | 1 + 16 files changed, 2095 insertions(+), 91 deletions(-) diff --git a/crates/socket-patch-cli/src/commands/get.rs b/crates/socket-patch-cli/src/commands/get.rs index e9d1bf94..60ffec69 100644 --- a/crates/socket-patch-cli/src/commands/get.rs +++ b/crates/socket-patch-cli/src/commands/get.rs @@ -683,6 +683,11 @@ pub struct DownloadParams { /// `--strict` forwarded to the nested apply (a beforeHash mismatch /// fails instead of warn-and-overwrite). pub strict: bool, + /// `--ecosystems` forwarded to the nested apply. Without this the + /// nested apply ran UNSCOPED over the whole manifest, so + /// `scan --ecosystems gem --sync` could mutate other ecosystems' + /// packages the user had explicitly filtered out. + pub ecosystems: Option>, /// Persist downloaded blob content into `.socket/blobs` (the apply /// flows need it for later hook/rollback runs). Vendor flows pass /// `false`: their patch content is staged in memory and the committed @@ -1098,6 +1103,7 @@ async fn run_nested_apply( download_mode: String, strict: bool, api: socket_patch_core::api::client::ApiClientEnvOverrides, + ecosystems: Option>, ) -> bool { // Apply re-resolves a relative manifest path against ITS `--cwd` // (`resolved_manifest_path`), but ours is already cwd-resolved — @@ -1119,6 +1125,11 @@ async fn run_nested_apply( api_token: api.api_token, org: api.org_slug, proxy_url: api.proxy_url, + // Scope the nested apply like the caller was scoped: leaving + // this at the default `None` made `scan --ecosystems gem --sync` + // apply the WHOLE manifest, mutating other ecosystems' packages + // the user filtered out. + ecosystems, ..crate::args::GlobalArgs::default() }, force: false, @@ -1232,10 +1243,7 @@ pub async fn download_and_apply_patches( // status/exit code degrade and it is never auto-applied. if files.is_empty() { if !params.json && !params.silent { - eprintln!( - " [fail] {} (patch has no applicable files)", - patch.purl - ); + eprintln!(" [fail] {} (patch has no applicable files)", patch.purl); } downloaded_patches.push(serde_json::json!({ "purl": patch.purl, @@ -1393,6 +1401,7 @@ pub async fn download_and_apply_patches( params.download_mode.clone(), params.strict, resolved_api_overrides(params), + params.ecosystems.clone(), ) .await; } @@ -1824,6 +1833,7 @@ pub async fn run(args: GetArgs) -> i32 { api_overrides: args.common.api_client_overrides(), all_releases: args.all_releases, strict: args.common.strict, + ecosystems: args.common.ecosystems.clone(), persist_blobs: true, }; @@ -2034,6 +2044,7 @@ async fn save_and_apply_patch(args: &GetArgs, patch: &PatchResponse) -> i32 { args.common.download_mode.clone(), args.common.strict, args.common.api_client_overrides(), + args.common.ecosystems.clone(), ) .await; } @@ -3106,7 +3117,10 @@ mod tests { // record — the guardrail-triggering condition the download/apply // flows now count as failed rather than applied. let mut broken = HashMap::new(); - broken.insert("src/lib.rs".to_string(), file_resp(Some(&"e".repeat(64)), None)); + broken.insert( + "src/lib.rs".to_string(), + file_resp(Some(&"e".repeat(64)), None), + ); let broken_patch = patch_with_files(broken); assert!( files_for_manifest(&broken_patch).is_empty(), diff --git a/crates/socket-patch-cli/src/commands/scan/mod.rs b/crates/socket-patch-cli/src/commands/scan/mod.rs index b76700ab..eefed677 100644 --- a/crates/socket-patch-cli/src/commands/scan/mod.rs +++ b/crates/socket-patch-cli/src/commands/scan/mod.rs @@ -411,6 +411,7 @@ fn download_params(args: &ScanArgs, save_only: bool, json: bool, silent: bool) - api_overrides: args.common.api_client_overrides(), all_releases: args.all_releases, strict: args.common.strict, + ecosystems: args.common.ecosystems.clone(), persist_blobs: args.mode != Some(ScanMode::Vendored), } } diff --git a/crates/socket-patch-cli/src/commands/scan/vendor_flow.rs b/crates/socket-patch-cli/src/commands/scan/vendor_flow.rs index 4851dad4..5927ed20 100644 --- a/crates/socket-patch-cli/src/commands/scan/vendor_flow.rs +++ b/crates/socket-patch-cli/src/commands/scan/vendor_flow.rs @@ -229,7 +229,9 @@ async fn run_scan_vendor_step( } /// The `scan --vendor` JSON path: discovery → (dry-run preview | download -/// → GC → vendor engine) → embedded VEX → print `result` → exit code. +/// → GC → vendor engine → embedded VEX) → print `result` → exit code. +/// The dry-run arm skips the VEX embed (emitting a `vex.skipped` marker +/// instead): a dry run vendors nothing, so there is no state to attest. /// /// Extracted from `run` (and called through `Box::pin`) so its sizeable /// temporaries get their own poll frame, entered only when `--vendor` is @@ -285,10 +287,17 @@ async fn run_vendor_json_path( ) .await; } - let final_code = - embed_vex_into_json(&args.common, &args.vex, manifest_path, 0, result).await; + // Embedded VEX is skipped on a dry run (apply.rs's precedent): + // nothing was vendored, so there is no just-vendored state to + // attest — generating here would verify the deliberately untouched + // tree (failing outright on a not-yet-vendored project) and write + // an attestation file during --dry-run. The marker keeps the + // request visible to JSON consumers instead of silently dropping it. + if args.vex.vex.is_some() { + result["vex"] = serde_json::json!({ "skipped": true, "reason": "dry_run" }); + } println!("{}", serde_json::to_string_pretty(&result).unwrap()); - return final_code; + return 0; } // 1) Download phase. Manifest mode reuses the `--apply` diff --git a/crates/socket-patch-cli/tests/docker_e2e_gem.rs b/crates/socket-patch-cli/tests/docker_e2e_gem.rs index c85450f9..e906f10d 100644 --- a/crates/socket-patch-cli/tests/docker_e2e_gem.rs +++ b/crates/socket-patch-cli/tests/docker_e2e_gem.rs @@ -9,6 +9,15 @@ //! - `gem_global_install_full_apply_chain` — `gem install` without //! --install-dir, installs to the system gem directory; socket-patch //! scans + applies with `--global`. +//! +//! The fixture serves the TRUE git-blob sha256 of the installed +//! `lib/colorize.rb` as `beforeHash` (computed once by a probe container +//! from the real upstream artifact — see [`upstream_before_hash`]), so +//! both apply paths run gated, without `--force`: `scan --sync`'s own +//! nested apply must patch the file in the same run, and the explicit +//! `apply` (against restored pristine bytes) must pass the variant gate. +//! With the old all-zeros placeholder the nested apply failed invisibly +//! and only the `--force` escape hatch was ever exercised. #![cfg(feature = "docker-e2e")] @@ -69,6 +78,52 @@ fn plain_sha256(content: &[u8]) -> String { hex::encode(hasher.finalize()) } +/// Probe: install colorize 1.1.0 from the real registry and emit the +/// git-blob sha256 of the exact `lib/colorize.rb` bytes `gem install` +/// lays down — the value the fixture must serve as `beforeHash` for the +/// default (no `--force`) apply path to pass the variant gate. +const BEFORE_HASH_PROBE_SCRIPT: &str = r#"#!/usr/bin/env bash +set -uo pipefail +gem install --no-document --install-dir /tmp/probe colorize -v 1.1.0 > /tmp/install.log 2>&1 || { + cat /tmp/install.log >&2; exit 1 +} +F=/tmp/probe/gems/colorize-1.1.0/lib/colorize.rb +[ -f "$F" ] || { echo "FAIL: $F missing" >&2; exit 1; } +{ printf 'blob %d\0' "$(wc -c < "$F")"; cat "$F"; } | sha256sum | cut -d' ' -f1 +"#; + +/// True git-blob sha256 of the colorize-1.1.0 `lib/colorize.rb` that +/// `gem install` produces, computed once per process by +/// [`BEFORE_HASH_PROBE_SCRIPT`] in a probe container. Serving this as the +/// fixture's `beforeHash` — instead of an all-zeros placeholder — is what +/// lets the gated apply paths run: with the placeholder, `scan --sync`'s +/// nested apply hit the variant gate and failed invisibly, so the chain +/// only ever proved the `--force` path. +fn upstream_before_hash() -> String { + static HASH: std::sync::OnceLock = std::sync::OnceLock::new(); + HASH.get_or_init(|| { + let out = run_container(BEFORE_HASH_PROBE_SCRIPT); + let stdout = String::from_utf8_lossy(&out.stdout); + let stderr = String::from_utf8_lossy(&out.stderr); + assert!( + out.status.success(), + "beforeHash probe container failed:\nstdout=\n{stdout}\nstderr=\n{stderr}" + ); + stdout + .lines() + .rev() + .map(str::trim) + .find(|l| l.len() == 64 && l.bytes().all(|b| b.is_ascii_hexdigit())) + .unwrap_or_else(|| { + panic!( + "beforeHash probe emitted no 64-hex git-blob sha256:\nstdout=\n{stdout}\nstderr=\n{stderr}" + ) + }) + .to_string() + }) + .clone() +} + /// Shared verification block for both scripts. Expects `GEM_FILE`, /// `EXPECTED_SHA`, and `APPLY_EXIT` to be set, plus the JSON captured in /// `/tmp/scan.json` and `/tmp/apply.json`. @@ -123,7 +178,7 @@ exit 0 "# } -async fn make_mock_server(after_hash: &str) -> MockServer { +async fn make_mock_server(before_hash: &str, after_hash: &str) -> MockServer { let listener = std::net::TcpListener::bind("0.0.0.0:0").expect("bind wiremock"); let server = MockServer::builder().listener(listener).start().await; @@ -169,9 +224,12 @@ async fn make_mock_server(after_hash: &str) -> MockServer { "publishedAt": "2024-01-01T00:00:00Z", "files": { // gem uses `package/` (npm-style) — apply strips - // the prefix and joins with the gem dir. + // the prefix and joins with the gem dir. beforeHash is + // the TRUE git-blob sha256 of the installed upstream + // file so the default (gated, no --force) apply path is + // what the chain exercises. "package/lib/colorize.rb": { - "beforeHash": "0000000000000000000000000000000000000000000000000000000000000000", + "beforeHash": before_hash, "afterHash": after_hash, "blobContent": blob_b64, } @@ -213,6 +271,9 @@ gem install --no-document --install-dir "$INSTALL_DIR" colorize -v 1.1.0 > /tmp/ GEM_FILE="$INSTALL_DIR/gems/colorize-1.1.0/lib/colorize.rb" [ -f "$GEM_FILE" ] || {{ echo "FAIL: $GEM_FILE missing" >&2; exit 1; }} echo "Installed to: $GEM_FILE" >&2 +# Keep a pristine copy: the explicit apply below is exercised against it +# after scan --sync's own nested apply has already patched the live file. +cp "$GEM_FILE" /tmp/pristine.rb # Pre-seed setup.manual so the agent-mode VEX leg keeps the gem patch through # property 7 (this project isn't `socket-patch setup`-configured; agent patches @@ -229,7 +290,19 @@ socket-patch scan --json --sync --yes \ --ecosystems gem > /tmp/scan.json 2>/tmp/sync.err cat /tmp/sync.err >&2 -socket-patch apply --json --force --offline --ecosystems gem > /tmp/apply.json 2>/tmp/apply.err +# The fixture serves the TRUE beforeHash, so scan --sync's own nested apply +# (which never uses --force) must pass the variant gate and patch the file +# in the same run — with an all-zeros placeholder this failed invisibly. +grep -q 'SOCKET-PATCH-E2E-MARKER' "$GEM_FILE" || {{ + echo "FAIL: scan --sync's nested apply left $GEM_FILE unpatched" >&2 + cat /tmp/scan.json >&2; head -3 "$GEM_FILE" >&2; exit 1; }} + +# Restore the pristine file so the explicit apply below exercises the +# default (gated, no --force) path end to end instead of short-circuiting +# on an already-patched file. +cp /tmp/pristine.rb "$GEM_FILE" + +socket-patch apply --json --offline --ecosystems gem > /tmp/apply.json 2>/tmp/apply.err APPLY_EXIT=$? cat /tmp/apply.err >&2 @@ -276,6 +349,9 @@ GEM_DIR=$(gem env gemdir) GEM_FILE="$GEM_DIR/gems/colorize-1.1.0/lib/colorize.rb" [ -f "$GEM_FILE" ] || {{ echo "FAIL: $GEM_FILE missing" >&2; exit 1; }} echo "Global-installed at: $GEM_FILE" >&2 +# Keep a pristine copy: the explicit apply below is exercised against it +# after scan --sync's own nested apply has already patched the live file. +cp "$GEM_FILE" /tmp/pristine.rb mkdir -p /workspace/proj && cd /workspace/proj @@ -285,7 +361,19 @@ socket-patch scan --json --sync --yes --global \ --ecosystems gem > /tmp/scan.json 2>/tmp/sync.err cat /tmp/sync.err >&2 -socket-patch apply --json --force --offline --global --ecosystems gem > /tmp/apply.json 2>/tmp/apply.err +# The fixture serves the TRUE beforeHash, so scan --sync's own nested apply +# (which never uses --force) must pass the variant gate and patch the file +# in the same run — with an all-zeros placeholder this failed invisibly. +grep -q 'SOCKET-PATCH-E2E-MARKER' "$GEM_FILE" || {{ + echo "FAIL: scan --sync's nested apply left $GEM_FILE unpatched" >&2 + cat /tmp/scan.json >&2; head -3 "$GEM_FILE" >&2; exit 1; }} + +# Restore the pristine file so the explicit apply below exercises the +# default (gated, no --force) path end to end instead of short-circuiting +# on an already-patched file. +cp /tmp/pristine.rb "$GEM_FILE" + +socket-patch apply --json --offline --global --ecosystems gem > /tmp/apply.json 2>/tmp/apply.err APPLY_EXIT=$? cat /tmp/apply.err >&2 {verify}"# @@ -390,12 +478,13 @@ async fn assert_api_path_exercised(server: &MockServer) { #[tokio::test] async fn gem_local_install_full_apply_chain() { - let after_hash = git_sha256(PATCHED_RB); - let server = make_mock_server(&after_hash).await; - let api_url = format!("http://host.docker.internal:{}", server.address().port()); if skip_if_no_image() { return; } + let before_hash = upstream_before_hash(); + let after_hash = git_sha256(PATCHED_RB); + let server = make_mock_server(&before_hash, &after_hash).await; + let api_url = format!("http://host.docker.internal:{}", server.address().port()); let expected_sha = plain_sha256(PATCHED_RB); let out = run_container(&local_script(&api_url, &expected_sha)); let stdout = String::from_utf8_lossy(&out.stdout); @@ -418,12 +507,13 @@ async fn gem_local_install_full_apply_chain() { #[tokio::test] async fn gem_global_install_full_apply_chain() { - let after_hash = git_sha256(PATCHED_RB); - let server = make_mock_server(&after_hash).await; - let api_url = format!("http://host.docker.internal:{}", server.address().port()); if skip_if_no_image() { return; } + let before_hash = upstream_before_hash(); + let after_hash = git_sha256(PATCHED_RB); + let server = make_mock_server(&before_hash, &after_hash).await; + let api_url = format!("http://host.docker.internal:{}", server.address().port()); let expected_sha = plain_sha256(PATCHED_RB); let out = run_container(&global_script(&api_url, &expected_sha)); let stdout = String::from_utf8_lossy(&out.stdout); diff --git a/crates/socket-patch-cli/tests/e2e_gem.rs b/crates/socket-patch-cli/tests/e2e_gem.rs index f9dc8120..9fb92a72 100644 --- a/crates/socket-patch-cli/tests/e2e_gem.rs +++ b/crates/socket-patch-cli/tests/e2e_gem.rs @@ -94,6 +94,11 @@ fn assert_run_ok(cwd: &Path, args: &[&str], context: &str) -> (String, String) { fn bundle_run(cwd: &Path, args: &[&str]) { let mut cmd = Command::new("bundle"); cmd.args(args).current_dir(cwd); + // Bundler 4 removed `bundle install --path`; BUNDLE_PATH is honored by + // bundler 2 through 4 and keeps the vendor/bundle/ruby/*/gems layout + // `find_gem_dir` expects. It also upholds cache_env's hermeticity + // invariant that every `bundle install` pins its gem tree to the fixture. + cmd.env("BUNDLE_PATH", "vendor/bundle"); cache_env::isolate(&mut cmd); let out = cmd.output().expect("failed to run bundle"); assert!( @@ -449,7 +454,7 @@ fn test_gem_full_lifecycle() { // -- Setup: create project and install activestorage@5.2.0 ---------------- write_gemfile(cwd); - bundle_run(cwd, &["install", "--path", "vendor/bundle"]); + bundle_run(cwd, &["install"]); let gem_dir = find_gem_dir(cwd); @@ -540,7 +545,7 @@ fn test_gem_dry_run() { let cwd = dir.path(); write_gemfile(cwd); - bundle_run(cwd, &["install", "--path", "vendor/bundle"]); + bundle_run(cwd, &["install"]); let gem_dir = find_gem_dir(cwd); @@ -579,7 +584,7 @@ fn test_gem_save_only() { let cwd = dir.path(); write_gemfile(cwd); - bundle_run(cwd, &["install", "--path", "vendor/bundle"]); + bundle_run(cwd, &["install"]); let gem_dir = find_gem_dir(cwd); diff --git a/crates/socket-patch-cli/tests/e2e_hosted_production.rs b/crates/socket-patch-cli/tests/e2e_hosted_production.rs index 1356509e..437b3092 100644 --- a/crates/socket-patch-cli/tests/e2e_hosted_production.rs +++ b/crates/socket-patch-cli/tests/e2e_hosted_production.rs @@ -558,6 +558,82 @@ async fn published_patch_dates(purl: &str) -> Result, Stri .unwrap_or_default()) } +/// `GET /patch/view/` against the real proxy — the same route the +/// CLI's free-proxy client fetches patch content from. Returns, per file the +/// patch touches, the `(path, beforeHash, afterHash)` triple (hashes are +/// git-blob sha256, `None` for pure additions/deletions respectively). +async fn published_patch_files( + uuid: &str, +) -> Result, Option)>, String> { + let url = format!("{PROXY}/patch/view/{uuid}"); + let resp = reqwest::Client::new() + .get(&url) + .header("Accept", "application/json") + .send() + .await + .map_err(|e| format!("GET {url}: {e}"))?; + let status = resp.status(); + let body = resp + .text() + .await + .map_err(|e| format!("GET {url}: reading body: {e}"))?; + if !status.is_success() { + return Err(format!("GET {url}: HTTP {status}\n{body}")); + } + let v: serde_json::Value = + serde_json::from_str(&body).map_err(|e| format!("GET {url}: bad JSON ({e}):\n{body}"))?; + Ok(v["files"] + .as_object() + .map(|m| { + m.iter() + .map(|(path, f)| { + ( + path.clone(), + f["beforeHash"].as_str().map(str::to_string), + f["afterHash"].as_str().map(str::to_string), + ) + }) + .collect() + }) + .unwrap_or_default()) +} + +/// The `sha256=` hex value the lock's CHECKSUMS section pins for +/// ` ()`, or `None` when the entry is absent. Bundler >= 2.6 +/// writes one two-space-indented ` name (version) sha256=` line per +/// resolved gem. +fn gem_lock_checksum(lock: &str, name: &str, version: &str) -> Option { + let prefix = format!(" {name} ({version}) sha256="); + lock.lines() + .find_map(|l| l.strip_prefix(&prefix).map(|h| h.trim().to_string())) +} + +/// Locate the bundler-installed `gems/-` directory under a +/// `BUNDLE_PATH` root. The `ruby/` segment in between varies by host +/// interpreter, so walk for it (depth-bounded — the layout is only a few +/// levels deep) instead of hardcoding the version. +fn installed_gem_dir(root: &Path, dir_name: &str, depth: usize) -> Option { + for entry in std::fs::read_dir(root).ok()?.flatten() { + let p = entry.path(); + if !p.is_dir() { + continue; + } + if p.file_name().is_some_and(|n| n == dir_name) + && p.parent() + .and_then(|d| d.file_name()) + .is_some_and(|n| n == "gems") + { + return Some(p); + } + if depth > 0 { + if let Some(found) = installed_gem_dir(&p, dir_name, depth - 1) { + return Some(found); + } + } + } + None +} + /// The Socket patch-registry base URL the gem rewriter pinned into `Gemfile` /// as `source "" do`, or `None` when no Socket source block is present. /// @@ -1621,6 +1697,19 @@ async fn gem_bundler_hosted_redirect_and_known_install_defect() { CHECKSUMS section)" ); } + // Capture the UPSTREAM checksum pin before any redirect. Bundler installs + // whatever matches this pin, so the redirect must replace it with the + // patched artifact's digest — the assertion after the redirect below is + // what makes an inert rewrite (URL repointed, upstream digest kept, so + // bundler verify-and-installs the UNPATCHED gem) go red instead of green. + let pristine_lock = read(&proj.join("Gemfile.lock")); + let upstream_sha = + gem_lock_checksum(&pristine_lock, GEM_NAME, GEM_VERSION).unwrap_or_else(|| { + panic!( + "{LEG}: `bundle lock --add-checksums` wrote no sha256 CHECKSUMS \ + entry for {GEM_NAME} ({GEM_VERSION}):\n{pristine_lock}" + ) + }); let install = tool(&proj, "bundle", &["install", "--quiet"], &env); if !ok(&install) { soft_skip!(LEG, "upstream `bundle install` failed:\n{}", dump(&install)); @@ -1642,6 +1731,19 @@ async fn gem_bundler_hosted_redirect_and_known_install_defect() { lock.contains("CHECKSUMS"), "{LEG}: Gemfile.lock lost its CHECKSUMS section:\n{lock}" ); + let redirected_sha = gem_lock_checksum(&lock, GEM_NAME, GEM_VERSION).unwrap_or_else(|| { + panic!( + "{LEG}: redirected Gemfile.lock carries no sha256 CHECKSUMS entry \ + for {GEM_NAME} ({GEM_VERSION}):\n{lock}" + ) + }); + assert_ne!( + redirected_sha, upstream_sha, + "{LEG}: the redirect left {GEM_NAME}'s CHECKSUMS pin at the UPSTREAM \ + sha256 — bundler would verify and install the unpatched artifact, so \ + the hosted patch is inert (the same blindspot that let an inert npm \ + patch stay green).\nGemfile.lock:\n{lock}" + ); // Known-broken leg: reinstall from the redirected Gemfile. std::fs::remove_dir_all(&bundle_path).ok(); @@ -1658,6 +1760,73 @@ async fn gem_bundler_hosted_redirect_and_known_install_defect() { appears to be FIXED — delete the tolerance branch in this test and \ assert unconditionally." ); + // Exit 0 proves only that bundler fetched an artifact matching the + // CHECKSUMS pin. Close the loop on CONTENT: fetch the patch's file + // manifest from the proxy and assert every file it rewrites landed + // on disk byte-exact (afterHash is the git-blob sha256 the patch + // service publishes — the same digest the CLI's apply verifies). + let patch_files = published_patch_files(GEM_UUID).await.unwrap_or_else(|e| { + panic!( + "{LEG}: `bundle install` from the redirected Gemfile succeeded \ + but the patch file manifest could not be fetched to verify \ + the installed content: {e}" + ) + }); + let gem_dir = installed_gem_dir( + Path::new(&bundle_path), + &format!("{GEM_NAME}-{GEM_VERSION}"), + 4, + ) + .unwrap_or_else(|| { + panic!( + "{LEG}: `bundle install` succeeded but no \ + gems/{GEM_NAME}-{GEM_VERSION} directory exists under \ + {bundle_path}" + ) + }); + use socket_patch_core::hash::git_sha256::compute_git_sha256_from_bytes; + let mut verified = 0usize; + let mut rewritten = 0usize; + for (path, before, after) in &patch_files { + // No afterHash = the patch deletes the file; nothing to hash. + let Some(after) = after else { continue }; + let rel = path.strip_prefix("package/").unwrap_or(path.as_str()); + let installed = gem_dir.join(rel); + let bytes = std::fs::read(&installed).unwrap_or_else(|e| { + panic!( + "{LEG}: patch {GEM_UUID} rewrites `{path}` but the \ + installed gem has no readable {}: {e}", + installed.display() + ) + }); + assert_eq!( + compute_git_sha256_from_bytes(&bytes), + *after, + "{LEG}: installed {} does not hash to the patch's afterHash — \ + bundler fetched an artifact whose content is NOT the \ + published patch", + installed.display() + ); + verified += 1; + if before.as_deref() != Some(after.as_str()) { + rewritten += 1; + } + } + assert!( + verified >= 1, + "{LEG}: patch {GEM_UUID} names no files with an afterHash, so \ + nothing was content-verified — the install success is vacuous" + ); + assert!( + rewritten >= 1, + "{LEG}: every file in patch {GEM_UUID} has afterHash == \ + beforeHash — the published patch is inert and this install \ + proved nothing" + ); + println!( + "{LEG}: verified {verified} patched file(s) on disk against the \ + published afterHash ({rewritten} differ from upstream)" + ); return; } let detail = dump(&reinstall); diff --git a/crates/socket-patch-cli/tests/in_process_get_manifest_path.rs b/crates/socket-patch-cli/tests/in_process_get_manifest_path.rs index e8c888cf..00fac6f9 100644 --- a/crates/socket-patch-cli/tests/in_process_get_manifest_path.rs +++ b/crates/socket-patch-cli/tests/in_process_get_manifest_path.rs @@ -18,6 +18,14 @@ //! `get --cwd proj ` made the nested apply look for //! `proj/proj/.socket/manifest.json`, hit the no-manifest clean no-op, //! and report success (`applied: 1`, exit 0) without patching anything. +//! +//! 3. `run_nested_apply` threaded cwd/global/silent/download-mode/strict +//! and the four API flags into the nested `ApplyArgs` but left +//! `ecosystems` at `GlobalArgs::default()` (`None`), so a download +//! scoped with `--ecosystems ` (`scan --ecosystems gem --sync`, or +//! `get --ecosystems npm `) ran its apply step UNSCOPED over the +//! whole manifest — mutating other ecosystems' packages the user had +//! explicitly filtered out. use std::path::Path; @@ -251,3 +259,176 @@ async fn get_with_relative_cwd_actually_applies() { against --cwd twice and apply no-op'd on a missing manifest" ); } + +// --------------------------------------------------------------------------- +// 4. --ecosystems must scope the nested apply, not just the download +// --------------------------------------------------------------------------- + +const COMPOSER_PURL: &str = "pkg:composer/acme/lib@1.0.0"; +const COMPOSER_UUID: &str = "44444444-4444-4444-8444-444444444444"; +const COMPOSER_ORIGINAL: &[u8] = b" (std::path::PathBuf, std::path::PathBuf) { + // The npm target package, installed with pre-patch content. + std::fs::write( + root.join("package.json"), + r#"{"name":"r","version":"0.0.0"}"#, + ) + .unwrap(); + let npm_pkg = root.join("node_modules/manifest-path-test"); + std::fs::create_dir_all(&npm_pkg).unwrap(); + std::fs::write( + npm_pkg.join("package.json"), + r#"{"name":"manifest-path-test","version":"1.0.0"}"#, + ) + .unwrap(); + std::fs::write(npm_pkg.join("index.js"), ORIGINAL).unwrap(); + + // A composer package installed side by side (the composer crawler + // probes `/vendor/composer/installed.json` plus a composer.json + // marker — fully deterministic, no tooling or env vars involved). + std::fs::write(root.join("composer.json"), "{}").unwrap(); + let composer_dir = root.join("vendor/composer"); + std::fs::create_dir_all(&composer_dir).unwrap(); + std::fs::write( + composer_dir.join("installed.json"), + r#"{"packages": [{"name": "acme/lib", "version": "1.0.0"}]}"#, + ) + .unwrap(); + let composer_pkg = root.join("vendor/acme/lib"); + std::fs::create_dir_all(&composer_pkg).unwrap(); + std::fs::write(composer_pkg.join("index.php"), COMPOSER_ORIGINAL).unwrap(); + + // Manifest pre-seeded with the PENDING composer patch record and its + // cached after-blob (no network fetch needed to apply it). + let composer_before_hash = git_sha256(COMPOSER_ORIGINAL); + let composer_after_hash = git_sha256(COMPOSER_PATCHED); + let socket = root.join(".socket"); + std::fs::create_dir_all(socket.join("blobs")).unwrap(); + std::fs::write( + socket.join("blobs").join(&composer_after_hash), + COMPOSER_PATCHED, + ) + .unwrap(); + std::fs::write( + socket.join("manifest.json"), + format!( + r#"{{ "patches": {{ + "{COMPOSER_PURL}": {{ + "uuid": "{COMPOSER_UUID}", + "exportedAt": "2024-01-01T00:00:00Z", + "files": {{ + "package/index.php": {{ + "beforeHash": "{composer_before_hash}", + "afterHash": "{composer_after_hash}" + }} + }}, + "vulnerabilities": {{}}, + "description": "pending composer patch", "license": "MIT", "tier": "free" + }} + }}}}"# + ), + ) + .unwrap(); + + (npm_pkg, composer_pkg) +} + +/// A download restricted with `--ecosystems npm` must run its nested +/// apply under the SAME scope: the pending composer patch staged by +/// [`stage_npm_and_composer_project`] must stay unapplied while the npm +/// package is patched. (The control test below proves the fixture bites — +/// an UNSCOPED run does patch the composer package.) +#[tokio::test] +#[serial] +async fn get_ecosystems_flag_scopes_nested_apply() { + let server = MockServer::start().await; + mount_view_mock(&server).await; + mount_search_mock(&server).await; + + let tmp = tempfile::tempdir().unwrap(); + let (npm_pkg, composer_pkg) = stage_npm_and_composer_project(tmp.path()); + + let mut args = get_args(PURL, tmp.path(), server.uri()); + args.save_only = false; // exercise the nested apply step + args.common.ecosystems = Some(vec!["npm".to_string()]); + + let code = run(args).await; + assert_eq!(code, 0, "scoped get + apply must succeed"); + + // The in-scope npm package was patched... + assert_eq!( + std::fs::read(npm_pkg.join("index.js")).unwrap(), + PATCHED, + "the npm package selected by --ecosystems npm must be patched" + ); + // ...and the out-of-scope composer package was NOT: patching it means + // `ecosystems` was dropped when building the nested ApplyArgs and the + // apply step ran unscoped over the whole manifest. + assert_eq!( + std::fs::read(composer_pkg.join("index.php")).unwrap(), + COMPOSER_ORIGINAL, + "--ecosystems npm must scope the nested apply: the pending \ + composer patch must not be applied" + ); +} + +/// Negative control for the scoped test above: WITHOUT `--ecosystems`, +/// the very same fixture's nested apply covers the whole manifest and +/// patches the pending composer package too. This pins that the fixture +/// genuinely can reach the composer package — if this stopped holding, +/// the scoped test would pass vacuously. +#[tokio::test] +#[serial] +async fn get_without_ecosystems_applies_whole_manifest() { + let server = MockServer::start().await; + mount_view_mock(&server).await; + mount_search_mock(&server).await; + + let tmp = tempfile::tempdir().unwrap(); + let (npm_pkg, composer_pkg) = stage_npm_and_composer_project(tmp.path()); + + let mut args = get_args(PURL, tmp.path(), server.uri()); + args.save_only = false; // exercise the nested apply step + + let code = run(args).await; + assert_eq!(code, 0, "unscoped get + apply must succeed"); + + assert_eq!( + std::fs::read(npm_pkg.join("index.js")).unwrap(), + PATCHED, + "the npm package must be patched" + ); + assert_eq!( + std::fs::read(composer_pkg.join("index.php")).unwrap(), + COMPOSER_PATCHED, + "without --ecosystems the nested apply covers the whole manifest, \ + including the pending composer patch" + ); +} diff --git a/crates/socket-patch-cli/tests/in_process_get_update_count.rs b/crates/socket-patch-cli/tests/in_process_get_update_count.rs index 04bb8ccf..6756f29a 100644 --- a/crates/socket-patch-cli/tests/in_process_get_update_count.rs +++ b/crates/socket-patch-cli/tests/in_process_get_update_count.rs @@ -73,6 +73,7 @@ fn params(root: &Path, server: &MockServer) -> DownloadParams { proxy_url: None, }, strict: false, + ecosystems: None, persist_blobs: true, // Skip release-narrowing; npm has no variants anyway. all_releases: true, diff --git a/crates/socket-patch-cli/tests/in_process_scan.rs b/crates/socket-patch-cli/tests/in_process_scan.rs index 622d2cb4..96cb0111 100644 --- a/crates/socket-patch-cli/tests/in_process_scan.rs +++ b/crates/socket-patch-cli/tests/in_process_scan.rs @@ -1433,3 +1433,110 @@ async fn scan_discovers_maven_and_nuget_in_every_mode() { } std::env::remove_var("MAVEN_REPO_LOCAL"); } + +// --------------------------------------------------------------------------- +// Regression: `scan --vendor --dry-run --vex` must skip the embedded VEX. +// +// The vendor JSON dry-run arm handed base_code 0 straight to +// `embed_vex_into_json`, which generated the document for real: on a +// not-yet-vendored project (no manifest) generation failed and flipped the +// whole dry run to exit 1; with attestable state it wrote the VEX file to +// disk — a mutation during --dry-run. Both must be skipped instead +// (apply.rs's precedent: a dry run has no just-vendored state to attest). +// --------------------------------------------------------------------------- + +#[tokio::test] +#[serial] +async fn scan_vendor_dry_run_with_vex_does_not_fail_on_not_yet_vendored() { + let server = MockServer::start().await; + mock_batch_one(&server).await; + mock_by_package(&server).await; + + let tmp = tempfile::tempdir().unwrap(); + write_root_package_json(tmp.path()); + write_npm_package(tmp.path(), "in-proc-scan", "1.0.0"); + + let vex_path = tmp.path().join("vendor-dry.vex.json"); + let mut args = default_args(tmp.path()); + args.common.api_url = Some(server.uri()); + args.vendor = true; + args.common.dry_run = true; + args.vex.vex = Some(vex_path.clone()); + + let code = run_scrubbed(args).await; + // Nothing is vendored yet (no manifest, no ledger) — that is the normal + // state a vendor dry run previews from, not an error. The old behavior + // generated the VEX for real, hit `manifest_not_found`, and exited 1. + assert_eq!( + code, 0, + "a vendor dry run must not fail for not-yet-vendored state" + ); + assert!(!vex_path.exists(), "dry run must not write the VEX file"); + // Prove the vendor dry-run path was actually entered (not + // short-circuited before discovery): batch + patch details were queried. + let reqs = recorded(&server).await; + assert!( + batch_posts(&reqs).len() == 1 && by_package_gets(&reqs) >= 1, + "vendor dry run must query batch + patch details; batch={}, by_package={}", + batch_posts(&reqs).len(), + by_package_gets(&reqs), + ); +} + +#[tokio::test] +#[serial] +async fn scan_vendor_dry_run_with_vex_does_not_write_attestation_file() { + let server = MockServer::start().await; + mock_batch_one(&server).await; + mock_by_package(&server).await; + + let tmp = tempfile::tempdir().unwrap(); + write_root_package_json(tmp.path()); + write_npm_package(tmp.path(), "in-proc-scan", "1.0.0"); + + // A manifest whose sole record WOULD attest successfully: vulnerability + // metadata for the statement, `setup.manual: ["npm"]` to pass the + // property-7 ecosystem filter, and `--vex-no-verify` below to skip the + // on-disk hash check. Under the old behavior the dry run generated the + // document for real and wrote it to disk. + let socket = tmp.path().join(".socket"); + std::fs::create_dir_all(&socket).unwrap(); + std::fs::write( + socket.join("manifest.json"), + r#"{ "patches": { + "pkg:npm/in-proc-scan@1.0.0": { + "uuid": "11111111-1111-4111-8111-111111111111", + "exportedAt": "2024-01-01T00:00:00Z", + "files": {}, + "vulnerabilities": { "GHSA-aaaa-bbbb-cccc": { + "cves": ["CVE-2024-0001"], "summary": "s", + "severity": "HIGH", "description": "d" + }}, + "description": "x", "license": "MIT", "tier": "free" + } + }, "setup": { "manual": ["npm"] } }"#, + ) + .unwrap(); + let before = std::fs::read_to_string(socket.join("manifest.json")).unwrap(); + + let vex_path = tmp.path().join("vendor-dry.vex.json"); + let mut args = default_args(tmp.path()); + args.common.api_url = Some(server.uri()); + args.vendor = true; + args.common.dry_run = true; + args.vex.vex = Some(vex_path.clone()); + args.vex.vex_no_verify = true; + + let code = run_scrubbed(args).await; + assert_eq!(code, 0, "vendor dry run with attestable state must exit 0"); + assert!( + !vex_path.exists(), + "dry run must not write the VEX file even when generation would succeed" + ); + // The dry run's non-mutation contract extends to the manifest. + let after = std::fs::read_to_string(socket.join("manifest.json")).unwrap(); + assert_eq!( + after, before, + "vendor dry run must leave the manifest byte-for-byte unchanged" + ); +} diff --git a/crates/socket-patch-cli/tests/setup_matrix_gem.rs b/crates/socket-patch-cli/tests/setup_matrix_gem.rs index 06711f6a..c8d68164 100644 --- a/crates/socket-patch-cli/tests/setup_matrix_gem.rs +++ b/crates/socket-patch-cli/tests/setup_matrix_gem.rs @@ -2,8 +2,20 @@ //! — it appends a managed `plugin "socket-patch"` block to the Gemfile and //! generates a committed in-tree Bundler plugin under `.socket/bundler-plugin/` //! whose `plugins.rb` re-runs `socket-patch apply --ecosystems gem` on every -//! `bundle install` (load-time digest gate + `after-install-all` hook). So the -//! with-setup cases are no longer a baseline gap. +//! `bundle install` (load-time digest gate + `after-install-all` hook). +//! +//! The with-setup Docker cases (`baseline_with_setup`, `alt_content_patchset`) +//! are still a [BASELINE GAP], for two structural reasons (verified +//! 2026-08-13): (a) installing the plugin evaluates `plugins.rb` BEFORE any +//! project gems land, and the load-time `SocketPatch.apply!` treats apply's +//! exit 1 ("No packages found that match available patches") as a genuine +//! failure and raises `Bundler::BundlerError`, so the FIRST `bundle install` +//! after `setup` on a never-installed project always dies — this is the gem +//! twin of the documented apply exit-semantics issue; (b) the matrix fixture's +//! synthetic beforeHashes only pass hash-gated ecosystems via npm's +//! mismatch-warn-and-apply path, which gem apply does not have. The +//! `after-install-all` re-apply flow itself works (asserted in-container with +//! realistic hashes). //! //! IMPORTANT — why this file carries a real assertion of its own: //! `smc::run_pm("gem", "bundler")` routes gem through the shared Docker diff --git a/crates/socket-patch-core/src/crawlers/ruby_crawler.rs b/crates/socket-patch-core/src/crawlers/ruby_crawler.rs index 8a6fed15..3d5e4892 100644 --- a/crates/socket-patch-core/src/crawlers/ruby_crawler.rs +++ b/crates/socket-patch-core/src/crawlers/ruby_crawler.rs @@ -22,7 +22,7 @@ impl RubyCrawler { /// Get gem installation paths based on options. /// - /// In local mode, checks `vendor/bundle/ruby/*/gems/` first (Bundler + /// In local mode, checks `vendor/bundle//*/gems/` first (Bundler /// deployment layout), then — only if the cwd holds a Bundler manifest /// or lockfile — falls back to the gem homes `gem env` reports. /// @@ -178,18 +178,31 @@ impl RubyCrawler { paths } - /// Find `vendor/bundle/ruby/*/gems/` directories. + /// Find `vendor/bundle//*/gems/` directories. + /// + /// Bundler's deployment scope is `#{Gem.ruby_engine}/#{ruby_version}` + /// (`Bundler.ruby_scope`) — `ruby` under MRI, but `jruby`/`truffleruby` + /// under the alternative engines. Hardcoding `ruby` made JRuby and + /// TruffleRuby deployments discover zero gems, so enumerate every + /// engine dir that holds `/gems/` children; non-engine + /// clutter is filtered by that shape. async fn get_vendor_bundle_paths(cwd: &Path) -> Vec { - let vendor_ruby = cwd.join("vendor").join("bundle").join("ruby"); + let vendor_bundle = cwd.join("vendor").join("bundle"); let mut paths = Vec::new(); - for entry in list_dir_entries(&vendor_ruby).await { - if !entry_is_dir(&entry).await { + for engine_entry in list_dir_entries(&vendor_bundle).await { + if !entry_is_dir(&engine_entry).await { continue; } - let gems_dir = vendor_ruby.join(entry.file_name()).join("gems"); - if is_dir(&gems_dir).await { - paths.push(gems_dir); + let engine_dir = vendor_bundle.join(engine_entry.file_name()); + for entry in list_dir_entries(&engine_dir).await { + if !entry_is_dir(&entry).await { + continue; + } + let gems_dir = engine_dir.join(entry.file_name()).join("gems"); + if is_dir(&gems_dir).await { + paths.push(gems_dir); + } } } paths @@ -338,22 +351,41 @@ impl RubyCrawler { /// /// Gem directories follow `-` (ruby-platform gems) or /// `--` (platform gems, e.g. - /// `nokogiri-1.16.5-x86_64-linux`). The name/version boundary is the - /// **first** `-` followed by a digit. A RubyGems version is dash-free - /// (prerelease dashes render as `.pre.`), so the version is the run up - /// to the next `-`; anything after that is the platform suffix, which - /// we drop — the installed platform is resolved later by hashing the - /// gem's files (the same model as PyPI's `artifact_id`). The qualified - /// `?platform=` PURL is only ever carried in the manifest/API. + /// `nokogiri-1.16.5-x86_64-linux`). A RubyGems version is dash-free + /// (prerelease dashes render as `.pre.`), so every `-` followed by a + /// digit is a candidate name/version boundary and the version is the + /// dash-free token after it; anything past that is the platform + /// suffix, which we drop — the installed platform is resolved later by + /// hashing the gem's files (the same model as PyPI's `artifact_id`). + /// The qualified `?platform=` PURL is only ever carried in the + /// manifest/API. + /// + /// Names may themselves contain `-` runs (`http-2`, + /// `http-2-next`), so the first candidate boundary is not always + /// right: `http-2-1.0.1` must parse as `("http-2", "1.0.1")`, not the + /// ghost `("http", "2")`. Real versions are almost always dotted while + /// digit runs embedded in names (`-2-`) and trailing platform OS + /// revisions (`-darwin-21`) are not, so prefer the LAST boundary whose + /// version token contains a `.`; fall back to the first dash-digit + /// boundary only when no dotted candidate exists (a bare + /// single-segment version like `g-1` is legal but vanishingly rare). fn parse_dir_name_version(dir_name: &str) -> Option<(String, String)> { - let idx = dir_name + let candidates: Vec = dir_name .match_indices('-') - .find(|(i, _)| dir_name[i + 1..].starts_with(|c: char| c.is_ascii_digit())) - .map(|(i, _)| i)?; - let name = &dir_name[..idx]; - let rest = &dir_name[idx + 1..]; + .filter(|(i, _)| dir_name[i + 1..].starts_with(|c: char| c.is_ascii_digit())) + .map(|(i, _)| i) + .collect(); // Version is the leading dash-free token; drop any `-`. - let version = rest.split('-').next().unwrap_or(rest); + let version_token = |i: usize| { + let rest = &dir_name[i + 1..]; + rest.split('-').next().unwrap_or(rest) + }; + let idx = *candidates + .iter() + .rfind(|&&i| version_token(i).contains('.')) + .or_else(|| candidates.first())?; + let name = &dir_name[..idx]; + let version = version_token(idx); if name.is_empty() || version.is_empty() { return None; } @@ -556,6 +588,34 @@ mod tests { assert_eq!(paths[0], vendor_gems); } + /// Bundler's deployment scope is `/` — `jruby` and + /// `truffleruby` deployments live beside `ruby` under `vendor/bundle` + /// and must be discovered too (hardcoding `ruby` found zero gems + /// there). Non-engine clutter — files, and dirs whose children hold no + /// `gems/` — must not produce paths. + #[tokio::test] + async fn test_get_vendor_bundle_paths_alternative_engines() { + let dir = tempfile::tempdir().unwrap(); + let bundle = dir.path().join("vendor").join("bundle"); + let ruby_gems = bundle.join("ruby").join("3.2.0").join("gems"); + let jruby_gems = bundle.join("jruby").join("3.1.4.0").join("gems"); + let truffle_gems = bundle.join("truffleruby").join("3.2.2").join("gems"); + for gems in [&ruby_gems, &jruby_gems, &truffle_gems] { + tokio::fs::create_dir_all(gems).await.unwrap(); + } + tokio::fs::write(bundle.join("install.log"), b"x") + .await + .unwrap(); + tokio::fs::create_dir_all(bundle.join("cache").join("3.2.0")) + .await + .unwrap(); + + let paths = RubyCrawler::get_vendor_bundle_paths(dir.path()).await; + assert_eq!(paths.len(), 3, "one gems dir per engine; got {paths:?}"); + let found: HashSet = paths.into_iter().collect(); + assert_eq!(found, HashSet::from([ruby_gems, jruby_gems, truffle_gems])); + } + #[tokio::test] async fn test_deduplication() { let dir = tempfile::tempdir().unwrap(); @@ -980,9 +1040,65 @@ mod tests { assert!(!is_safe_gem_coordinate("rails", "C:1.0.0")); } + /// Names with embedded `-` runs (`http-2`, `http-2-next`) must + /// keep the digits in the name: the boundary is the LAST dash-digit + /// whose version token is dotted, not the first dash-digit. Without + /// that preference `http-2-1.0.1` parsed as `("http", "2")` — a ghost + /// PURL — and the real gem was never discovered. + #[test] + fn parse_dir_name_version_prefers_last_dotted_boundary() { + assert_eq!( + RubyCrawler::parse_dir_name_version("http-2-1.0.1"), + Some(("http-2".to_string(), "1.0.1".to_string())) + ); + assert_eq!( + RubyCrawler::parse_dir_name_version("http-2-next-1.0.3"), + Some(("http-2-next".to_string(), "1.0.3".to_string())) + ); + // A platform suffix after the real version still drops. + assert_eq!( + RubyCrawler::parse_dir_name_version("http-2-1.0.1-java"), + Some(("http-2".to_string(), "1.0.1".to_string())) + ); + } + + /// The dotted-boundary preference must not regress the plain shapes: + /// dotted versions, prereleases, platform dirs, and — via the + /// first-boundary fallback — bare single-segment versions (legal per + /// RubyGems, just vanishingly rare). + #[test] + fn parse_dir_name_version_boundary_shapes() { + assert_eq!( + RubyCrawler::parse_dir_name_version("rack-3.1.0"), + Some(("rack".to_string(), "3.1.0".to_string())) + ); + assert_eq!( + RubyCrawler::parse_dir_name_version("aws-sdk-s3-1.140.0"), + Some(("aws-sdk-s3".to_string(), "1.140.0".to_string())) + ); + assert_eq!( + RubyCrawler::parse_dir_name_version("gem2-1.0"), + Some(("gem2".to_string(), "1.0".to_string())) + ); + // No dotted candidate → first dash-digit boundary fallback. + assert_eq!( + RubyCrawler::parse_dir_name_version("g-1"), + Some(("g".to_string(), "1".to_string())) + ); + // Prerelease dashes render as dots, so the token stays dotted. + assert_eq!( + RubyCrawler::parse_dir_name_version("rails-7.1.0.beta1"), + Some(("rails".to_string(), "7.1.0.beta1".to_string())) + ); + assert_eq!( + RubyCrawler::parse_dir_name_version("nokogiri-1.16.0-arm64-darwin"), + Some(("nokogiri".to_string(), "1.16.0".to_string())) + ); + } + /// Gem names with embedded underscores/digits and multi-dash names - /// must keep their full name; the version starts at the first - /// dash-then-digit boundary. + /// must keep their full name; the version starts at the dash-then-digit + /// boundary that opens the dotted version token. #[test] fn parse_dir_name_version_name_shapes() { assert_eq!( diff --git a/crates/socket-patch-core/src/patch/redirect/mod.rs b/crates/socket-patch-core/src/patch/redirect/mod.rs index 0ff40eb5..87740fcf 100644 --- a/crates/socket-patch-core/src/patch/redirect/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/mod.rs @@ -1673,6 +1673,31 @@ pub(crate) fn gem_line_trailing_options(tail: &str) -> String { } } +/// The source-selecting option a `gem` line's argument tail carries, if any +/// (only the code before any `#` comment counts). Bundler allows ONE source +/// per gem, so an option like `git:` preserved into the Socket source block +/// OVERRIDES the block and the redirect becomes a silent no-op. Mirrors the +/// token list `vendor::gem::rest_blocks_edit` refuses for the same reason. +fn gem_tail_source_option(tail: &str) -> Option<&'static str> { + let code = tail.split('#').next().unwrap_or(""); + [ + "path:", + ":path", + "git:", + ":git", + "github:", + ":github", + "source:", + ":source", + "gist:", + ":gist", + "bitbucket:", + ":bitbucket", + ] + .into_iter() + .find(|tok| code.contains(tok)) +} + fn rewrite_gem( files: &BTreeMap, overrides: &[DepOverride], @@ -1713,35 +1738,159 @@ fn rewrite_gem( continue; }; + // Platform-suffixed CHECKSUMS siblings (`name (version-arm64-darwin) + // sha256=`) mean bundler resolves platform-specific gems the patch + // registry does not serve — redirecting would pin the bare-platform + // sha while installs keep fetching the upstream platform gem + // (guaranteed mismatch or a silently unpatched install). Fail closed: + // skip the dep entirely. + if let Some(lk) = lock.as_deref() { + let platform_re = Regex::new( + &(String::from(r"(?m)^ ") + + ®ex::escape(&dep.name) + + r" \(" + + ®ex::escape(&dep.version) + + r"-[^)]+\) sha256="), + ) + .unwrap(); + if platform_re.is_match(lk) { + result.warnings.push(RewriteWarning { + code: "redirect_gem_platform_unsupported".into(), + detail: format!( + "Gemfile.lock CHECKSUMS carries platform-specific entries for {} {} — \ + the patch registry serves only the ruby platform gem; redirect skipped", + dep.name, dep.version + ), + }); + continue; + } + } + + // Whether THIS dep's Gemfile source redirect is in place (just + // written or already present) — the lock pin below is gated on it. + let mut source_placed = false; if let Some(gf) = gemfile.as_mut() { - if !gf.contains(&format!("source \"{}\"", ov.index_url)) { + // Grant-agnostic idempotency guard: the grant-token (and patch + // uuid) segments of the index URL rotate per request, so an + // exact-URL check misses the block a previous run wrote and this + // run would wrap the gem line inside it — nesting source blocks. + // Wildcard the rotating segments instead (mirrors the CHECKSUMS + // at-target guard below). + let mut url_pat = regex::escape(&ov.index_url); + for rotating in [&dep.token, &dep.patch_uuid] { + if !rotating.is_empty() { + url_pat = + url_pat.replace(®ex::escape(&format!("/{rotating}/")), "/[^/\"]+/"); + } + } + let block_re = Regex::new( + &(String::from(r#"(?m)^source "("#) + + &url_pat + + r#")" do\n gem ["']"# + + ®ex::escape(&dep.name) + + r#"["']"#), + ) + .unwrap(); + if let Some(m) = block_re.captures(gf) { + let url = m.get(1).unwrap(); + if url.as_str() == ov.index_url { + source_placed = true; + } else { + // Rotated grant: refresh the URL in place — never nest. + let (range, old_url) = (url.range(), url.as_str().to_string()); + gf.replace_range(range, &ov.index_url); + gemfile_changed = true; + result.edits.push(FileEdit { + path: "Gemfile".into(), + kind: "redirect_gemfile_source_url".into(), + action: "rewritten".into(), + key: Some(dep.name.clone()), + original: Some(Value::String(old_url)), + new: Some(Value::String(ov.index_url.clone())), + }); + source_placed = true; + } + } else { + // Tolerate the legal spellings of a declaration: tab / extra + // spaces after `gem`, and the parenthesized call form. let gem_line_re = Regex::new( - &(String::from(r#"(?m)^\s*gem ["']"#) + &(String::from(r#"(?m)^\s*gem(?:[ \t]*(\()[ \t]*|[ \t]+)["']"#) + ®ex::escape(&dep.name) + r#"["']([^\n]*)$"#), ) .unwrap(); - let block = format!( - "source \"{}\" do\n gem \"{}\", \"{}\"\nend", - ov.index_url, dep.name, dep.version - ); + // Looser "declared at all?" probe: gates the append branch — + // appending next to a declaration the recognizer above cannot + // parse would leave the gem declared twice (bundler + // hard-fails on the duplicate). + let declared_re = Regex::new( + &(String::from(r#"(?m)^[ \t]*gem\b[^\n]*["']"#) + + ®ex::escape(&dep.name) + + r#"["']"#), + ) + .unwrap(); if let Some(m) = gem_line_re.captures(gf) { + let range = m.get(0).unwrap().range(); let original = m.get(0).unwrap().as_str().to_string(); + let paren = m.get(1).is_some(); + let raw_tail = m.get(2).unwrap().as_str().to_string(); + // A parenthesized call keeps its closing `)` in the tail: + // strip it (dropping any comment with it), or fail closed + // when it is absent (the call continues past this line). + let tail = if paren { + let code = raw_tail.split('#').next().unwrap_or("").trim_end(); + match code.strip_suffix(')') { + Some(t) => t.to_string(), + None => { + result.warnings.push(RewriteWarning { + code: "redirect_gem_unrecognized_declaration".into(), + detail: format!( + "the `gem \"{}\"` declaration is in a form the \ + rewriter cannot safely edit; redirect skipped", + dep.name + ), + }); + continue; + } + } + } else { + raw_tail + }; + // A source-selecting option would move into the block and + // OVERRIDE it in bundler's DSL, leaving the redirect a + // silent no-op that still gets attested. Fail closed. + if let Some(tok) = gem_tail_source_option(&tail) { + result.warnings.push(RewriteWarning { + code: "redirect_gem_source_option".into(), + detail: format!( + "the `gem \"{}\"` declaration carries `{tok}`, which would \ + override the Socket source block; redirect skipped", + dep.name + ), + }); + continue; + } // Trailing options (`require: false`, `group: …`) must // survive the move into the source block — dropping // `require: false` auto-requires the gem at boot. - let opts = gem_line_trailing_options(m.get(1).unwrap().as_str()); + let opts = gem_line_trailing_options(&tail); let block = if opts.is_empty() { - block + format!( + "source \"{}\" do\n gem \"{}\", \"{}\"\nend", + ov.index_url, dep.name, dep.version + ) } else { format!( "source \"{}\" do\n gem \"{}\", \"{}\", {opts}\nend", ov.index_url, dep.name, dep.version ) }; - // Plain replacen: the block may carry user text (`opts`), - // which a regex replacement would `$`-expand. - *gf = gf.replacen(&original, &block, 1); + // Splice by the match's byte range: a substring replace of + // the line's TEXT would hit an identical commented-out + // duplicate earlier in the file and corrupt it (and the + // block may carry user text a regex replacement would + // `$`-expand). + gf.replace_range(range, &block); gemfile_changed = true; result.edits.push(FileEdit { path: "Gemfile".into(), @@ -1751,7 +1900,23 @@ fn rewrite_gem( original: Some(Value::String(original)), new: Some(Value::String(block)), }); + source_placed = true; + } else if declared_re.is_match(gf) { + result.warnings.push(RewriteWarning { + code: "redirect_gem_unrecognized_declaration".into(), + detail: format!( + "the `gem \"{}\"` declaration is in a form the rewriter \ + cannot safely edit; redirect skipped", + dep.name + ), + }); + continue; } else { + // Genuinely undeclared (a transitive dep): append a block. + let block = format!( + "source \"{}\" do\n gem \"{}\", \"{}\"\nend", + ov.index_url, dep.name, dep.version + ); let sep = if gf.ends_with('\n') { "" } else { "\n" }; *gf = format!("{gf}{sep}{block}\n"); gemfile_changed = true; @@ -1763,17 +1928,32 @@ fn rewrite_gem( original: None, new: Some(Value::String(block)), }); + source_placed = true; } } } if let Some(lk) = lock.as_mut() { + // The pin only makes sense once the source redirect is in place + // (just written or already present): pinning the patched sha + // while the gem still resolves upstream guarantees a checksum + // failure on the next install. + if !source_placed { + result.warnings.push(RewriteWarning { + code: "redirect_gem_lock_without_source".into(), + detail: format!( + "no Gemfile source redirect is in place for {} — CHECKSUMS pin skipped", + dep.name + ), + }); + continue; + } let sum_line_re = Regex::new( &(String::from(r"(?m)^( ") + ®ex::escape(&dep.name) + r" \(" + ®ex::escape(&dep.version) - + r"\)) sha256=[0-9a-f]+$"), + + r"\)) sha256=([0-9a-f]+)$"), ) .unwrap(); let new_val = format!("{} ({}) sha256={sha256}", dep.name, dep.version); @@ -1781,7 +1961,15 @@ fn rewrite_gem( // target value; recording an edit would grow the ledger forever. if lk.contains(&format!("\n {new_val}\n")) || lk.ends_with(&format!("\n {new_val}")) { // no-op - } else if sum_line_re.is_match(lk) { + } else if let Some(m) = sum_line_re.captures(lk) { + // The pre-edit line goes into the ledger as `original` so a + // future `--revert` can restore the upstream sha. + let old_val = format!( + "{} ({}) sha256={}", + dep.name, + dep.version, + m.get(2).unwrap().as_str() + ); *lk = sum_line_re .replace(lk, format!("${{1}} sha256={sha256}").as_str()) .to_string(); @@ -1791,7 +1979,7 @@ fn rewrite_gem( kind: "redirect_gemfile_lock_checksum".into(), action: "rewritten".into(), key: Some(dep.name.clone()), - original: None, + original: Some(Value::String(old_val)), new: Some(Value::String(new_val)), }); } else if checksums_re.is_match(lk) { @@ -1826,6 +2014,21 @@ fn rewrite_gem( } } + // The rewritten pair breaks bundler's frozen/deployment mode: the lock's + // GEM section still records the upstream source, so `bundle install` with + // `frozen`/`--deployment` set rejects the Gemfile's new source block. + // Mirror of the CLI's pnpm trust-lockfile warning. + if gemfile_changed || lock_changed { + result.warnings.push(RewriteWarning { + code: "redirect_gem_frozen_install".into(), + detail: "Gemfile was repointed at the Socket patch registry but Gemfile.lock's \ + GEM section still records the upstream source; bundler rejects the pair \ + under frozen/deployment mode — run `bundle install` (unfrozen) once to \ + record the new source in Gemfile.lock" + .into(), + }); + } + if gemfile_changed { if let Some(gf) = gemfile { result.files.insert("Gemfile".into(), gf); @@ -3586,6 +3789,338 @@ mod tests { ); } + /// A minimal Gemfile.lock with the given CHECKSUMS lines (rails 7.0.0). + fn gem_lock(checksums: &str) -> String { + format!( + "GEM\n remote: https://rubygems.org/\n specs:\n rails (7.0.0)\n\n\ + PLATFORMS\n ruby\n\nDEPENDENCIES\n rails (= 7.0.0)\n\n\ + CHECKSUMS\n{checksums}\n\nBUNDLED WITH\n 2.6.2\n" + ) + } + + /// The edit must splice by the regex match's byte range: a substring + /// replace of the matched line's TEXT finds an identical commented-out + /// duplicate earlier in the file first and corrupts the comment while the + /// live line keeps resolving upstream. + #[test] + fn gemfile_rewrite_ignores_commented_duplicate() { + let mut files = BTreeMap::new(); + files.insert( + "Gemfile".to_string(), + "source \"https://rubygems.org\"\n\ + # gem \"rails\", \"7.0.0\" pinned during the 6.x upgrade\n\ + gem \"rails\", \"7.0.0\"\n" + .to_string(), + ); + let r = rewrite_registry_redirect(&files, &[gem_override("rails", "7.0.0")]); + let out = r.files.get("Gemfile").expect("Gemfile rewritten"); + assert!( + out.contains("\n# gem \"rails\", \"7.0.0\" pinned during the 6.x upgrade\n"), + "commented-out duplicate left untouched: {out}" + ); + assert!( + out.contains( + "\nsource \"https://patch.test/gem/tok/uuid/\" do\n gem \"rails\", \"7.0.0\"\nend\n" + ), + "live line replaced by the source block: {out}" + ); + } + + /// The grant token in the index URL rotates per request, so a re-run must + /// recognize the source block a previous run wrote (token-wildcard match, + /// not exact URL) and refresh its URL in place — never wrap the block's + /// gem line inside a new nested block. + #[test] + fn gemfile_rerun_with_rotated_grant_updates_url_never_nests() { + fn ov(token: &str) -> DepOverride { + let mut o = gem_override("rails", "7.0.0"); + o.token = token.into(); + if let Some(r) = o.registry_override.as_mut() { + r.index_url = format!("https://patch.test/gem/{token}/uuid/"); + } + o + } + let mut files = BTreeMap::new(); + files.insert( + "Gemfile".to_string(), + "source \"https://rubygems.org\"\n\ngem \"rails\", \"7.0.0\"\n".to_string(), + ); + let first = rewrite_registry_redirect(&files, &[ov("tok-one")]); + let redirected = first.files.get("Gemfile").expect("first run rewrites"); + files.insert("Gemfile".to_string(), redirected.clone()); + + let second = rewrite_registry_redirect(&files, &[ov("tok-two")]); + let out = second + .files + .get("Gemfile") + .expect("rotated grant refreshes the URL"); + assert_eq!( + out.matches("source \"https://patch.test/gem/").count(), + 1, + "exactly one Socket source block, never nested: {out}" + ); + assert!( + out.contains( + "source \"https://patch.test/gem/tok-two/uuid/\" do\n gem \"rails\", \"7.0.0\"\nend" + ), + "URL refreshed in place: {out}" + ); + assert!(!out.contains("tok-one"), "old grant token gone: {out}"); + assert!( + second + .edits + .iter() + .any(|e| e.kind == "redirect_gemfile_source_url" + && e.original + == Some(Value::String("https://patch.test/gem/tok-one/uuid/".into()))), + "URL refresh recorded with the old URL as original: {:?}", + second.edits + ); + + // Same grant again: a true no-op. + files.insert("Gemfile".to_string(), out.clone()); + let third = rewrite_registry_redirect(&files, &[ov("tok-two")]); + assert!( + third.files.is_empty() && third.edits.is_empty(), + "same-grant re-run must be a no-op: files={:?} edits={:?}", + third.files.keys(), + third.edits + ); + } + + /// A gem-level source option (`git:` / `path:` / `github:` / `source:`) + /// preserved into the Socket source block OVERRIDES it in bundler's DSL, + /// leaving the redirect a silent no-op that still gets attested. Fail + /// closed: warn and leave both files untouched. + #[test] + fn gemfile_gem_with_source_option_fails_closed() { + let mut files = BTreeMap::new(); + files.insert( + "Gemfile".to_string(), + "source \"https://rubygems.org\"\n\n\ + gem \"rails\", \"7.0.0\", git: \"https://github.com/rails/rails\"\n" + .to_string(), + ); + files.insert( + "Gemfile.lock".to_string(), + gem_lock(&format!(" rails (7.0.0) sha256={}", "2".repeat(64))), + ); + let r = rewrite_registry_redirect(&files, &[gem_override("rails", "7.0.0")]); + assert!( + r.files.is_empty() && r.edits.is_empty(), + "source-selecting option must skip the redirect: files={:?} edits={:?}", + r.files.keys(), + r.edits + ); + assert!( + r.warnings + .iter() + .any(|w| w.code == "redirect_gem_source_option"), + "skip must warn: {:?}", + r.warnings + ); + } + + /// Platform-specific CHECKSUMS siblings (`rails (7.0.0-arm64-darwin)`) + /// mean bundler resolves a platform gem the patch registry does not + /// serve — the bare-platform pin would leave the platform line at the + /// upstream sha (or duplicate the bare line). Fail closed: skip the dep. + #[test] + fn gem_platform_checksums_fail_closed() { + let mut files = BTreeMap::new(); + files.insert( + "Gemfile".to_string(), + "source \"https://rubygems.org\"\n\ngem \"rails\", \"7.0.0\"\n".to_string(), + ); + files.insert( + "Gemfile.lock".to_string(), + gem_lock(&format!( + " rails (7.0.0) sha256={}\n rails (7.0.0-arm64-darwin) sha256={}", + "2".repeat(64), + "3".repeat(64) + )), + ); + let r = rewrite_registry_redirect(&files, &[gem_override("rails", "7.0.0")]); + assert!( + r.files.is_empty() && r.edits.is_empty(), + "platform gems must skip the whole dep: files={:?} edits={:?}", + r.files.keys(), + r.edits + ); + assert!( + r.warnings + .iter() + .any(|w| w.code == "redirect_gem_platform_unsupported"), + "skip must warn: {:?}", + r.warnings + ); + } + + /// Legal-but-non-canonical declarations (parenthesized call, tab / double + /// space after `gem`) must be recognized and rewritten in place — falling + /// through to the append branch declares the gem twice, which bundler + /// rejects. + #[test] + fn gemfile_paren_and_whitespace_declarations_are_rewritten_not_duplicated() { + let mut files = BTreeMap::new(); + files.insert( + "Gemfile".to_string(), + "source \"https://rubygems.org\"\n\n\ + gem(\"rails\", \"7.0.0\", require: false)\n\ + gem\t\"puma\", \"6.0.0\"\n" + .to_string(), + ); + let r = rewrite_registry_redirect( + &files, + &[ + gem_override("rails", "7.0.0"), + gem_override("puma", "6.0.0"), + ], + ); + let out = r.files.get("Gemfile").expect("Gemfile rewritten"); + assert!( + out.contains( + "source \"https://patch.test/gem/tok/uuid/\" do\n \ + gem \"rails\", \"7.0.0\", require: false\nend" + ), + "paren declaration rewritten with options kept, `)` stripped: {out}" + ); + assert!( + !out.contains("gem(\"rails\"") && !out.contains("gem\t\"puma\""), + "original declarations replaced, not duplicated: {out}" + ); + assert!( + out.contains( + "source \"https://patch.test/gem/tok/uuid/\" do\n gem \"puma\", \"6.0.0\"\nend" + ), + "tab-separated declaration rewritten: {out}" + ); + } + + /// A declaration the recognizer cannot parse (`gem\"rails\"` — legal ruby, + /// no separator) must NOT fall through to the append branch: warn and skip + /// instead of declaring the gem twice. + #[test] + fn gemfile_unrecognizable_declaration_fails_closed_no_append() { + let mut files = BTreeMap::new(); + files.insert( + "Gemfile".to_string(), + "source \"https://rubygems.org\"\n\ngem\"rails\", \"7.0.0\"\n".to_string(), + ); + let r = rewrite_registry_redirect(&files, &[gem_override("rails", "7.0.0")]); + assert!( + r.files.is_empty() && r.edits.is_empty(), + "unrecognizable declaration must not append a duplicate: files={:?} edits={:?}", + r.files.keys(), + r.edits + ); + assert!( + r.warnings + .iter() + .any(|w| w.code == "redirect_gem_unrecognized_declaration"), + "skip must warn: {:?}", + r.warnings + ); + } + + /// The CHECKSUMS pin is gated on the Gemfile source redirect being in + /// place: with no Gemfile in the candidate map, pinning the patched sha + /// while the gem still resolves upstream guarantees a checksum failure. + #[test] + fn gem_lock_pin_gated_on_source_redirect() { + let mut files = BTreeMap::new(); + files.insert( + "Gemfile.lock".to_string(), + gem_lock(&format!(" rails (7.0.0) sha256={}", "2".repeat(64))), + ); + let r = rewrite_registry_redirect(&files, &[gem_override("rails", "7.0.0")]); + assert!( + r.files.is_empty() && r.edits.is_empty(), + "lock pin without a source redirect must be skipped: files={:?} edits={:?}", + r.files.keys(), + r.edits + ); + assert!( + r.warnings + .iter() + .any(|w| w.code == "redirect_gem_lock_without_source"), + "skip must warn: {:?}", + r.warnings + ); + } + + /// A landed gem redirect breaks bundler frozen/deployment installs (the + /// lock's GEM section still records the upstream source), so the rewrite + /// must say so — and only when it actually changed something. + #[test] + fn gem_redirect_warns_about_frozen_installs() { + let mut files = BTreeMap::new(); + files.insert( + "Gemfile".to_string(), + "source \"https://rubygems.org\"\n\ngem \"rails\", \"7.0.0\"\n".to_string(), + ); + files.insert( + "Gemfile.lock".to_string(), + gem_lock(&format!(" rails (7.0.0) sha256={}", "2".repeat(64))), + ); + let ovr = gem_override("rails", "7.0.0"); + let first = rewrite_registry_redirect(&files, std::slice::from_ref(&ovr)); + assert!( + first + .warnings + .iter() + .any(|w| w.code == "redirect_gem_frozen_install"), + "landed redirect must warn about frozen installs: {:?}", + first.warnings + ); + + // No-op re-run: nothing landed, so no frozen-install warning. + for (name, content) in first.files { + files.insert(name, content); + } + let second = rewrite_registry_redirect(&files, std::slice::from_ref(&ovr)); + assert!( + second.files.is_empty() + && !second + .warnings + .iter() + .any(|w| w.code == "redirect_gem_frozen_install"), + "a no-op re-run must not warn: files={:?} warnings={:?}", + second.files.keys(), + second.warnings + ); + } + + /// The rewritten CHECKSUMS edit must carry the pre-edit line as + /// `original` — with `None` the ledger cannot restore the upstream sha on + /// a future revert. + #[test] + fn gem_lock_rewrite_records_original_checksum_line() { + let mut files = BTreeMap::new(); + files.insert( + "Gemfile".to_string(), + "source \"https://rubygems.org\"\n\ngem \"rails\", \"7.0.0\"\n".to_string(), + ); + files.insert( + "Gemfile.lock".to_string(), + gem_lock(&format!(" rails (7.0.0) sha256={}", "2".repeat(64))), + ); + let r = rewrite_registry_redirect(&files, &[gem_override("rails", "7.0.0")]); + let edit = r + .edits + .iter() + .find(|e| e.kind == "redirect_gemfile_lock_checksum" && e.action == "rewritten") + .expect("lock checksum edit recorded"); + assert_eq!( + edit.original, + Some(Value::String(format!( + "rails (7.0.0) sha256={}", + "2".repeat(64) + ))), + "pre-edit CHECKSUMS line captured for revert" + ); + } + /// An unparseable package-lock.json must surface a warning, not silently /// skip the npm redirect entirely (missing-lockfile already warns; a /// corrupt lockfile is strictly worse and was silent). diff --git a/crates/socket-patch-core/src/vendor/gem.rs b/crates/socket-patch-core/src/vendor/gem.rs index 3f3ee42b..2e5c47a2 100644 --- a/crates/socket-patch-core/src/vendor/gem.rs +++ b/crates/socket-patch-core/src/vendor/gem.rs @@ -38,7 +38,11 @@ //! trailing options like `require: false` preserved) or, for a transitive //! dependency, a managed block appended at EOF. Anything //! the conservative line grammar cannot prove safe to rewrite is REFUSED — -//! never guessed at. +//! never guessed at. The one exception is OUR OWN previous wiring: a patch +//! update moves the manifest to a new uuid (same purl), and a `path:` that +//! parses as the socket vendor dir for exactly this gem is repointed in +//! place (the older patch uuid is re-vendored automatically, like the +//! npm/cargo/golang backends — no revert-first). //! //! The stub gemspec from `/specifications/` is copied into the //! vendored dir as `.gemspec` (a path source needs one; the spike showed @@ -63,7 +67,7 @@ use super::common::{ already_patched_result, copy_matches_after_hashes, done, refused, service_offline_conflict, synthesized_result, }; -use super::path::vendor_uuid_dir_rel; +use super::path::{parse_vendor_path, vendor_uuid_dir_rel}; use super::registry_fetch::extract_gem_data; use super::service_fetch::{ fetch_verified_archive, fetch_verified_secondary, SecondaryArtifactResult, ServiceArtifact, @@ -459,15 +463,50 @@ pub async fn vendor_gem( original: None, new: Some(Value::String(block.clone())), }, + // Re-vendor over our own wiring (see `GemfilePlan::RewireOurs`): + // `original: None`, carried forward by the caller. The managed-fence + // form stays `Added` with the whole updated block so revert deletes + // the fence too. + GemfilePlan::RewireOurs { + new_line, + managed_block, + .. + } => match managed_block { + Some(block) => WiringRecord { + file: GEMFILE.to_string(), + kind: GEMFILE_WIRING_KIND.to_string(), + action: WiringAction::Added, + key: Some(name.to_string()), + original: None, + new: Some(Value::String(block.clone())), + }, + None => WiringRecord { + file: GEMFILE.to_string(), + kind: GEMFILE_WIRING_KIND.to_string(), + action: WiringAction::Rewritten, + key: Some(name.to_string()), + original: None, + new: Some(Value::String(new_line.clone())), + }, + }, + }; + // A rewire lifted OUR OWN previous PATH section, not pre-vendor + // fragments: record `original: None` — the true originals live in the + // ledger entry being replaced, which the caller carries forward by + // wiring identity (`persist_vendor_entry`). + let lock_original = if lock_edit.rewired_ours { + None + } else { + let mut original_lines: Vec = lock_edit + .removed_spec_block + .iter() + .map(|l| Value::String(l.clone())) + .collect(); + if let Some(dep) = &lock_edit.old_dep_line { + original_lines.push(Value::String(dep.clone())); + } + Some(Value::Array(original_lines)) }; - let mut original_lines: Vec = lock_edit - .removed_spec_block - .iter() - .map(|l| Value::String(l.clone())) - .collect(); - if let Some(dep) = &lock_edit.old_dep_line { - original_lines.push(Value::String(dep.clone())); - } let mut new_lines: Vec = lock_edit .path_section .iter() @@ -479,7 +518,7 @@ pub async fn vendor_gem( kind: LOCK_WIRING_KIND.to_string(), action: WiringAction::Rewritten, key: Some(name.to_string()), - original: Some(Value::Array(original_lines)), + original: lock_original, new: Some(Value::Array(new_lines)), }; let mut wiring = vec![gemfile_record, lock_record]; @@ -496,6 +535,23 @@ pub async fn vendor_gem( original: Some(Value::String(orig_line.clone())), new: Some(Value::String(new_line.clone())), }); + } else if lock_edit.rewired_ours { + // Re-vendor with the bare path-form line already in place (our + // previous run stripped the registry token): the record must ride + // again with `original: None` — dropped, the first run's registry + // `sha256=` line would vanish from the ledger with the entry being + // replaced, and a later --revert could no longer restore it (a bare + // leftover on a registry gem hard-fails frozen installs, exit 16). + if let Some(bare) = &lock_edit.checksum_bare { + wiring.push(WiringRecord { + file: GEMFILE_LOCK.to_string(), + kind: LOCK_CHECKSUM_WIRING_KIND.to_string(), + action: WiringAction::Rewritten, + key: Some(name.to_string()), + original: None, + new: Some(Value::String(bare.clone())), + }); + } } let entry = VendorEntry { @@ -882,6 +938,22 @@ enum GemfilePlan { original_line: String, new_line: String, }, + /// The declaration already carries OUR OWN `path:` wiring from an older + /// patch uuid (a patch update changes the uuid, never the purl): + /// repoint it at the new copy in place, everything else on the line + /// byte-preserved. The wiring record carries `original: None` — the true + /// pre-vendor line lives in the ledger entry being replaced and the + /// caller carries it forward by wiring identity (`persist_vendor_entry`); + /// recording the old-uuid line would make a later revert "restore" a + /// dangling vendor pointer. `managed_block` is `Some(updated block)` when + /// the line sits inside our managed fence (the transitive-gem form): the + /// record then stays `Added` with the whole block, so revert still + /// deletes the fence. + RewireOurs { + original_line: String, + new_line: String, + managed_block: Option, + }, /// The gem is transitive (not declared): append a fenced managed block. Append { block: String }, } @@ -893,7 +965,9 @@ enum GemfilePlan { /// (inside a `group`/`platforms`/conditional block), parenthesized, /// continued onto the next line, conditional, or already carrying a /// `path:`/`git:`/`github:` source — is refused rather than guessed at: a -/// wrong Gemfile rewrite executes on every `bundle` invocation. +/// wrong Gemfile rewrite executes on every `bundle` invocation. The one +/// `path:` exception is our own vendored dir for this gem (an older patch +/// uuid), which is repointed in place — see [`GemfilePlan::RewireOurs`]. fn plan_gemfile_edit( text: &str, name: &str, @@ -935,6 +1009,30 @@ fn plan_gemfile_edit( "the `gem \"{name}\"` declaration uses a parenthesized call" )); } + // Our own wiring from an older patch uuid: the `path:` value parses as + // the socket vendor dir for exactly this gem. Repoint it in place — + // refusing here (the source-option blocklist below) would make every + // patch update demand a manual `vendor --revert` first. A path that + // parses as anything else (a user fork, another gem's dir) still refuses. + if let Some(prev_rel) = gem_line_path_value(&rest) { + if is_our_vendor_rel(prev_rel, name, version) { + let original_line = lines[idx].to_string(); + // The rel appears exactly once (its charset excludes quotes and + // `#`, and the code before `path:` cannot contain a `/`-bearing + // token); swapping just the value preserves quote style and + // trailing options verbatim. + let new_line = original_line.replacen(prev_rel, rel, 1); + let managed_block = (idx > 0 + && lines[idx - 1] == MANAGED_OPEN + && lines.get(idx + 1).is_some_and(|l| *l == MANAGED_CLOSE)) + .then(|| format!("{MANAGED_OPEN}\n{new_line}\n{MANAGED_CLOSE}\n")); + return Ok(GemfilePlan::RewireOurs { + original_line, + new_line, + managed_block, + }); + } + } if let Some(reason) = rest_blocks_edit(&rest) { return Err(format!( "the `gem \"{name}\"` declaration is not editable: {reason}" @@ -1025,11 +1123,45 @@ fn rest_blocks_edit(rest: &str) -> Option { None } +/// The quoted `path:` option value on a gem line's argument tail (only the +/// code before any `#` comment counts) — the form our own rewrite emits. +/// `None` for anything else (`:path =>`, interpolation, no `path:` at all): +/// those fall through to [`rest_blocks_edit`]'s refusal, fail-closed. +fn gem_line_path_value(rest: &str) -> Option<&str> { + let code = rest.split('#').next().unwrap_or(""); + let idx = code.find("path:")?; + if idx > 0 && !matches!(code.as_bytes()[idx - 1], b' ' | b'\t' | b',') { + return None; + } + let after = code[idx + "path:".len()..].trim_start(); + let q = after.chars().next()?; + if q != '"' && q != '\'' { + return None; + } + let value = &after[1..]; + let end = value.find(q)?; + Some(&value[..end]) +} + +/// True when a `path:`/`remote:` value is OUR vendored dir for exactly this +/// gem (`.socket/vendor/gem//-`) — the shape +/// [`vendor_gem`] wires, and the only wiring a patch UPDATE (new uuid, same +/// purl) may rewire. +fn is_our_vendor_rel(value: &str, name: &str, version: &str) -> bool { + parse_vendor_path(value) + .is_some_and(|p| p.eco == "gem" && p.leaf == format!("{name}-{version}")) +} + fn apply_gemfile_plan(text: &str, plan: &GemfilePlan) -> String { match plan { GemfilePlan::Rewrite { original_line, new_line, + } + | GemfilePlan::RewireOurs { + original_line, + new_line, + .. } => { let mut lines: Vec<&str> = text.split('\n').collect(); if let Some(i) = lines.iter().position(|l| *l == original_line) { @@ -1068,6 +1200,17 @@ struct LockEdit { /// "original" — reverting it onto a registry-sourced lock would break /// frozen installs). checksum_rewrite: Option<(String, String)>, + /// The spec block was lifted from OUR OWN previous PATH section (a + /// re-vendor to a newer patch uuid), not from GEM/specs: the lifted + /// fragments are this backend's own prior wiring, so the caller records + /// `original: None` and the true pre-vendor originals ride forward from + /// the ledger entry being replaced (`persist_vendor_entry`). + rewired_ours: bool, + /// The already-bare CHECKSUMS line for the gem, when one is present. + /// Only consulted on a re-vendor (`rewired_ours`): the checksum record + /// must ride again or the first run's registry `sha256=` restore line + /// drops out of the ledger with the entry being replaced. + checksum_bare: Option, } /// Produce the pair-edited lock text (see the module doc for the canonical @@ -1077,21 +1220,86 @@ struct LockEdit { fn edit_lock(text: &str, name: &str, version: &str, rel: &str) -> Result { let mut lines: Vec = text.split('\n').map(str::to_string).collect(); - // 1. Lift the gem's spec block out of GEM/specs. + // 1. Lift the gem's spec block out of GEM/specs — or, on a re-vendor to + // a newer patch uuid (same purl), out of the PATH section our previous + // run emitted. let (gem_start, gem_end) = section_span(&lines, "GEM").ok_or_else(|| "Gemfile.lock has no GEM section".to_string())?; if !(gem_start..gem_end).any(|i| lines[i] == " specs:") { return Err("Gemfile.lock GEM section has no specs: stanza".to_string()); } - let target = format!(" {name} ({version})"); - let block_start = (gem_start..gem_end) - .find(|&i| lines[i] == target) - .ok_or_else(|| format!("Gemfile.lock GEM specs has no entry `{name} ({version})`"))?; - let mut block_end = block_start + 1; - while block_end < gem_end && lines[block_end].starts_with(" ") { - block_end += 1; + // SECURITY/fail-closed: platform-suffixed installs were refused + // (`platform_gem_unsupported`) before this point, so a platform-suffixed + // GEM spec sibling means the lock disagrees with the installed tree — + // and lifting only the plain entry would leave the sibling behind as a + // stale registry spec. The CHECKSUMS branch below refuses the same + // shape, but only bundler ≥ 2.6 locks have a CHECKSUMS section to catch + // it in. + let platform_prefix = format!("{version}-"); + for line in lines.iter().take(gem_end).skip(gem_start + 1) { + if let Some((n, v)) = spec_entry(line) { + if n == name && v.starts_with(&platform_prefix) { + return Err(format!( + "Gemfile.lock GEM specs has a platform-suffixed entry `{n} ({v})` but the installed gem is not platform-specific; the lock disagrees with the install (re-resolve it before vendoring)" + )); + } + } } - let removed_spec_block: Vec = lines.drain(block_start..block_end).collect(); + let target = format!(" {name} ({version})"); + let mut rewired_ours = false; + let removed_spec_block: Vec = match (gem_start..gem_end).find(|&i| lines[i] == target) { + Some(block_start) => { + let mut block_end = block_start + 1; + while block_end < gem_end && lines[block_end].starts_with(" ") { + block_end += 1; + } + lines.drain(block_start..block_end).collect() + } + None => { + // Re-vendor: the entry lives in the PATH section our previous + // run emitted (remote parses as our vendored dir for exactly + // this gem). Lift the block and drop the old section — step 3 + // re-emits it at the NEW uuid's sorted position. The lifted + // lines are our own wiring, not pre-vendor originals: flagged + // via `rewired_ours` (see the `LockEdit` field docs). + let Some((ps, pe)) = find_our_path_section(&lines, name, version) else { + return Err(format!( + "Gemfile.lock GEM specs has no entry `{name} ({version})`" + )); + }; + let block_start = (ps..pe).find(|&i| lines[i] == target).ok_or_else(|| { + format!( + "Gemfile.lock PATH section for `{name}` lost its `{name} ({version})` spec entry" + ) + })?; + let mut block_end = block_start + 1; + while block_end < pe && lines[block_end].starts_with(" ") { + block_end += 1; + } + // Grammar-strict: besides the block, the section must be exactly + // what vendor wrote (header, one remote, specs:, blank + // separators). Anything extra — a hand edit, a merged-in second + // spec — would be destroyed by the drain below; never guess. + let non_block: Vec<&str> = (ps..pe) + .filter(|i| !(block_start..block_end).contains(i)) + .map(|i| lines[i].as_str()) + .filter(|l| !l.is_empty()) + .collect(); + if non_block.len() != 3 + || non_block[0] != "PATH" + || !non_block[1].starts_with(" remote: ") + || non_block[2] != " specs:" + { + return Err(format!( + "Gemfile.lock PATH section for `{name} ({version})` is not the shape vendor wrote; refusing to rewire it" + )); + } + let block: Vec = lines[block_start..block_end].to_vec(); + lines.drain(ps..pe); + rewired_ours = true; + block + } + }; // 2. DEPENDENCIES: exact pin + `!` path-source marker. A transitive gem // (absent pre-vendor) is inserted at bundler's sorted position — it is a @@ -1122,8 +1330,16 @@ fn edit_lock(text: &str, name: &str, version: &str, rel: &str) -> Result lines.insert(insert_at, new_dep_line.clone()), } - // 3. PATH section directly above the GEM section (bundler's canonical - // placement; spike claim 2). `remote:` is the bare relative path. + // 3. PATH section above the GEM section, at bundler's SORTED position + // among any existing PATH sections: bundler emits path/git/plugin + // sources sorted by identifier (source_list.rb `lock_other_sources`, + // verified against bundler 4.0.15) — `source at ``` for a path + // source, so PATH sections order by their remote path and all sit in one + // contiguous run (no other source's identifier can start with that + // prefix). Splicing at invocation order instead churns the committed + // lock on the next `bundle lock`. Non-PATH leading sections keep the + // legacy insert-before-GEM fallback. `remote:` is the bare relative + // path (spike claim 2). let mut path_section = vec![ "PATH".to_string(), format!(" remote: {rel}"), @@ -1134,9 +1350,29 @@ fn edit_lock(text: &str, name: &str, version: &str, rel: &str) -> Result our_ident => { + at = i; + break; + } + // Ours sorts after this section (a remote-less section is + // grammar-degenerate; keep the legacy after-everything spot). + _ => at = end.min(gem_hdr), + } + i = end; + } else { + i += 1; + } + } let mut insert = path_section.clone(); - insert.push(String::new()); // blank separator before GEM - lines.splice(gem_hdr..gem_hdr, insert); + insert.push(String::new()); // blank separator before the next section + lines.splice(at..at, insert); // 4. CHECKSUMS (bundler ≥ 2.6 `lockfile_checksums`): a path-sourced gem // keeps a BARE ` ()` entry — bundler's own re-lock emits @@ -1147,9 +1383,9 @@ fn edit_lock(text: &str, name: &str, version: &str, rel: &str) -> Result = None; + let mut checksum_bare: Option = None; if let Some((ck_start, ck_end)) = section_span(&lines, "CHECKSUMS") { let bare = format!(" {name} ({version})"); - let platform_prefix = format!("{version}-"); let mut plain_at: Option = None; for (i, line) in lines.iter().enumerate().take(ck_end).skip(ck_start + 1) { match checksum_entry(line) { @@ -1190,6 +1426,8 @@ fn edit_lock(text: &str, name: &str, version: &str, rel: &str) -> Result Result Result Option<(usize, usize)> { let start = lines.iter().position(|l| l.as_str() == header)?; + Some((start, section_end(lines, start))) +} + +/// End (exclusive) of the section whose column-0 header sits at `start` — +/// the [`section_span`] rule for a known header position. +fn section_end(lines: &[String], start: usize) -> usize { let mut end = start + 1; while end < lines.len() { let l = &lines[end]; @@ -1217,7 +1463,39 @@ fn section_span(lines: &[String], header: &str) -> Option<(usize, usize)> { } end += 1; } - Some((start, end)) + end +} + +/// Bundler's lock-sort identifier for a path source — `source at ``` +/// (`Source::Path#to_s`, aliased as `identifier`); sections order by a +/// byte-wise comparison of these, which Rust's `str` ordering matches. +fn path_source_identifier(path: &str) -> String { + format!("source at `{path}`") +} + +/// The ` remote: ` value of the section slice starting at its header line. +fn path_section_remote(section: &[String]) -> Option<&str> { + section.iter().find_map(|l| l.strip_prefix(" remote: ")) +} + +/// Find the PATH section whose `remote:` is OUR vendored dir for this gem — +/// any patch uuid (the previous run's wiring, sought during a re-vendor). +fn find_our_path_section(lines: &[String], name: &str, version: &str) -> Option<(usize, usize)> { + let mut i = 0; + while i < lines.len() { + if lines[i].as_str() == "PATH" { + let end = section_end(lines, i); + if path_section_remote(&lines[i..end]) + .is_some_and(|p| is_our_vendor_rel(p, name, version)) + { + return Some((i, end)); + } + i = end; + } else { + i += 1; + } + } + None } /// Name of a 2-space DEPENDENCIES entry (` rack (~> 3.1)` / ` rack!`). @@ -1239,6 +1517,25 @@ fn spec_entry_name(line: &str) -> Option<&str> { Some(rest.split(' ').next().unwrap_or(rest)) } +/// Parse a 4-space specs entry line: ` ()`, nothing after +/// the closing paren. Returns `(name, parenthesized token)` — the platform +/// suffix stays inside the token, mirroring [`checksum_entry`]'s grammar at +/// specs indentation (` ffi (1.17.2-aarch64-linux-gnu)`). +fn spec_entry(line: &str) -> Option<(&str, &str)> { + let rest = line.strip_prefix(" ")?; + if rest.is_empty() || rest.starts_with(' ') { + return None; + } + let open = rest.find(" (")?; + let after = &rest[open + 2..]; + let close = after.find(')')?; + let (name, ver, tail) = (&rest[..open], &after[..close], &after[close + 1..]); + if name.is_empty() || ver.is_empty() || !tail.is_empty() { + return None; + } + Some((name, ver)) +} + /// Parse a CHECKSUMS entry line: two-space indent, ` ()` or /// ` (-)`, then optional space-separated tokens /// (`sha256=` on registry entries, nothing on path entries). Returns @@ -1665,10 +1962,21 @@ mod tests { installed: &Path, record: &PatchRecord, dry_run: bool, + ) -> VendorOutcome { + run_vendor_purl(PURL, root, blobs, installed, record, dry_run).await + } + + async fn run_vendor_purl( + purl: &str, + root: &Path, + blobs: &Path, + installed: &Path, + record: &PatchRecord, + dry_run: bool, ) -> VendorOutcome { let sources = PatchSources::blobs_only(blobs); vendor_gem( - PURL, + purl, installed, root, record, @@ -1681,6 +1989,23 @@ mod tests { .await } + /// Simulate the CLI caller's `persist_vendor_entry` carry-forward: fill + /// the replacement entry's `original: None` holes from the entry being + /// replaced, by wiring identity (file, kind, key). + fn carry_forward_originals(prev: &VendorEntry, next: &mut VendorEntry) { + for rec in &mut next.wiring { + if rec.action == WiringAction::Rewritten && rec.original.is_none() { + if let Some(p) = prev + .wiring + .iter() + .find(|p| p.file == rec.file && p.kind == rec.kind && p.key == rec.key) + { + rec.original = p.original.clone(); + } + } + } + } + fn expected_lock_direct() -> String { format!( "PATH\n remote: {rel}\n specs:\n rack (3.2.6)\n base64 (>= 0.1.0)\n\nGEM\n remote: https://rubygems.org/\n specs:\n puma (6.4.2)\n nio4r (~> 2.0)\n\nPLATFORMS\n arm64-darwin-23\n ruby\n\nDEPENDENCIES\n puma\n rack (= 3.2.6)!\n\nBUNDLED WITH\n 2.5.22\n", @@ -2777,6 +3102,360 @@ mod tests { ); } + // ── multiple vendored gems: PATH sections sort like bundler's ──────────── + + /// Second gem for multi-PATH tests. Its uuid sorts BEFORE rack's + /// (`1a…` < `9f…`), so vendoring rack first is the order a naive + /// insert-before-GEM splice would leave unsorted. + const UUID_PUMA: &str = "1a2b3c4d-5e6f-4a1b-8c2d-3e4f5a6b7c8d"; + const PURL_PUMA: &str = "pkg:gem/puma@6.4.2"; + const PRISTINE_PUMA: &[u8] = b"module Puma\n VERSION = \"6.4.2\"\nend\n"; + const PATCHED_PUMA: &[u8] = + b"module Puma\n SOCKET_PATCHED = true\n VERSION = \"6.4.2\"\nend\n"; + const GEMSPEC_PUMA: &str = "Gem::Specification.new do |s|\n s.name = \"puma\"\n s.version = \"6.4.2\"\n s.require_paths = [\"lib\"]\nend\n"; + + fn puma_rel() -> String { + format!(".socket/vendor/gem/{UUID_PUMA}/puma-6.4.2") + } + + /// Add a puma install + blob + record alongside [`fixture`]'s rack, so a + /// test can vendor TWO gems into one project. + async fn add_puma_fixture(installed_rack: &Path, blobs: &Path) -> (PathBuf, PatchRecord) { + let gems = installed_rack.parent().unwrap(); + let installed = gems.join("puma-6.4.2"); + tokio::fs::create_dir_all(installed.join("lib")) + .await + .unwrap(); + tokio::fs::write(installed.join("lib/puma.rb"), PRISTINE_PUMA) + .await + .unwrap(); + let specs = gems.parent().unwrap().join("specifications"); + tokio::fs::write(specs.join("puma-6.4.2.gemspec"), GEMSPEC_PUMA) + .await + .unwrap(); + let before = compute_git_sha256_from_bytes(PRISTINE_PUMA); + let after = compute_git_sha256_from_bytes(PATCHED_PUMA); + tokio::fs::write(blobs.join(&after), PATCHED_PUMA) + .await + .unwrap(); + let mut files = HashMap::new(); + files.insert( + "lib/puma.rb".to_string(), + PatchFileInfo { + before_hash: before, + after_hash: after, + }, + ); + let record = PatchRecord { + uuid: UUID_PUMA.to_string(), + exported_at: "2026-06-09T00:00:00Z".to_string(), + files, + vulnerabilities: HashMap::new(), + description: String::new(), + license: String::new(), + tier: String::new(), + }; + (installed, record) + } + + fn expected_lock_two_path() -> String { + format!( + "PATH\n remote: {puma}\n specs:\n puma (6.4.2)\n nio4r (~> 2.0)\n\nPATH\n remote: {rack}\n specs:\n rack (3.2.6)\n base64 (>= 0.1.0)\n\nGEM\n remote: https://rubygems.org/\n specs:\n\nPLATFORMS\n arm64-darwin-23\n ruby\n\nDEPENDENCIES\n puma (= 6.4.2)!\n rack (= 3.2.6)!\n\nBUNDLED WITH\n 2.5.22\n", + puma = puma_rel(), + rack = copy_rel() + ) + } + + /// Bundler regenerates PATH sections sorted by source identifier — by + /// remote path, the uuid level deciding here (verified against a real + /// bundler 4.0.15 `bundle lock` over this exact two-PATH shape). The + /// splice must land each new section at that sorted position no matter + /// the vendor invocation order, or the committed lock churns on the + /// next `bundle lock`/`bundle install`. + #[tokio::test] + async fn test_two_path_sections_sorted_regardless_of_vendor_order() { + for rack_first in [true, false] { + let (_tmp, root, installed_rack, blobs, record_rack) = + fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; + let (installed_puma, record_puma) = add_puma_fixture(&installed_rack, &blobs).await; + let runs: [(&str, &Path, &PatchRecord); 2] = if rack_first { + [ + (PURL, &installed_rack, &record_rack), + (PURL_PUMA, &installed_puma, &record_puma), + ] + } else { + [ + (PURL_PUMA, &installed_puma, &record_puma), + (PURL, &installed_rack, &record_rack), + ] + }; + for (purl, installed, record) in runs { + let (result, _e, _w) = unwrap_done( + run_vendor_purl(purl, &root, &blobs, installed, record, false).await, + ); + assert!(result.success, "vendor {purl} failed: {:?}", result.error); + } + let lock = tokio::fs::read_to_string(root.join(GEMFILE_LOCK)) + .await + .unwrap(); + assert_eq!(lock, expected_lock_two_path(), "rack_first={rack_first}"); + } + } + + // ── re-vendor: a patch update (new uuid, same purl) ────────────────────── + + /// Re-vendor uuid; sorts BEFORE `UUID_PUMA`'s (`0e…` < `1a…`). + const UUID2: &str = "0e1f2a3b-4c5d-4e6f-8a7b-9c0d1e2f3a4b"; + + /// A patch update moves the manifest to a NEW uuid for the same gem. The + /// CLI re-vendors straight over the first run's live wiring (originals + /// carried forward and the old uuid dir swept by the caller — no + /// revert-first; the cargo backend pins the same design). Both pair + /// files must be repointed in place, with `original: None` on the + /// rewired records. + #[tokio::test] + async fn test_revendor_new_uuid_direct_rewires_in_place() { + let (_tmp, root, installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; + let (r1, e1, _) = unwrap_done(run_vendor(&root, &blobs, &installed, &record, false).await); + assert!(r1.success); + let entry1 = e1.unwrap(); + + let mut record2 = record.clone(); + record2.uuid = UUID2.to_string(); + let (r2, e2, _) = unwrap_done(run_vendor(&root, &blobs, &installed, &record2, false).await); + assert!(r2.success, "re-vendor must succeed: {:?}", r2.error); + + let new_rel = format!(".socket/vendor/gem/{UUID2}/rack-3.2.6"); + assert_eq!( + tokio::fs::read_to_string(root.join(GEMFILE)).await.unwrap(), + format!( + "source \"https://rubygems.org\"\n\ngem \"puma\"\ngem \"rack\", \"3.2.6\", path: \"{new_rel}\"\n" + ), + "Gemfile repointed in place" + ); + assert_eq!( + tokio::fs::read_to_string(root.join(GEMFILE_LOCK)) + .await + .unwrap(), + expected_lock_direct().replace(UUID, UUID2), + "lock repointed in place" + ); + // New copy built; the old uuid dir is left for the caller's + // stale-artifact sweep (the caller owns the ledger). + assert_eq!( + tokio::fs::read(root.join(&new_rel).join("lib/rack.rb")) + .await + .unwrap(), + PATCHED + ); + assert!(root.join(format!(".socket/vendor/gem/{UUID}")).exists()); + + // The rewired records carry `original: None` — never the old-uuid + // lines (reverting those would "restore" a dangling vendor pointer). + let mut entry2 = e2.expect("re-vendor emits the new ledger entry"); + assert_eq!(entry2.uuid, UUID2); + assert_eq!(entry2.wiring.len(), 2); + for rec in &entry2.wiring { + assert_eq!(rec.action, WiringAction::Rewritten); + assert!(rec.original.is_none(), "{rec:?}"); + } + + // With the caller's carry-forward applied, revert restores the + // PRE-VENDOR files byte-exactly. + carry_forward_originals(&entry1, &mut entry2); + let outcome = revert_gem(&entry2, &root, false).await; + assert!(outcome.success, "{:?}", outcome.error); + assert!( + !outcome + .warnings + .iter() + .any(|w| w.code == "vendor_lock_entry_drifted"), + "clean revert must not report drift: {:?}", + outcome.warnings + ); + assert_eq!( + tokio::fs::read_to_string(root.join(GEMFILE)).await.unwrap(), + GEMFILE_DIRECT + ); + assert_eq!( + tokio::fs::read_to_string(root.join(GEMFILE_LOCK)) + .await + .unwrap(), + LOCK_DIRECT + ); + } + + /// Transitive form: the managed block is repointed in place (never + /// duplicated) and the record stays `Added` with the WHOLE updated block, + /// so a later revert deletes the fence. + #[tokio::test] + async fn test_revendor_new_uuid_transitive_updates_managed_block() { + let (_tmp, root, installed, blobs, record) = + fixture(GEMFILE_TRANSITIVE, LOCK_TRANSITIVE).await; + let (r1, e1, _) = unwrap_done(run_vendor(&root, &blobs, &installed, &record, false).await); + assert!(r1.success); + let entry1 = e1.unwrap(); + + let mut record2 = record.clone(); + record2.uuid = UUID2.to_string(); + let (r2, e2, _) = unwrap_done(run_vendor(&root, &blobs, &installed, &record2, false).await); + assert!(r2.success, "re-vendor must succeed: {:?}", r2.error); + + let new_rel = format!(".socket/vendor/gem/{UUID2}/rack-3.2.6"); + let new_block = format!( + "{MANAGED_OPEN}\ngem \"rack\", \"3.2.6\", path: \"{new_rel}\"\n{MANAGED_CLOSE}\n" + ); + assert_eq!( + tokio::fs::read_to_string(root.join(GEMFILE)).await.unwrap(), + format!("source \"https://rubygems.org\"\n\ngem \"puma\"\n{new_block}"), + "ONE managed block, repointed — never a duplicate declaration" + ); + + let mut entry2 = e2.unwrap(); + assert_eq!(entry2.wiring[0].action, WiringAction::Added); + assert!(entry2.wiring[0].original.is_none()); + assert_eq!( + entry2.wiring[0].new.as_ref().unwrap(), + &Value::String(new_block) + ); + + carry_forward_originals(&entry1, &mut entry2); + let outcome = revert_gem(&entry2, &root, false).await; + assert!(outcome.success, "{:?}", outcome.error); + assert!( + !outcome + .warnings + .iter() + .any(|w| w.code == "vendor_lock_entry_drifted"), + "clean revert must not report drift: {:?}", + outcome.warnings + ); + assert_eq!( + tokio::fs::read_to_string(root.join(GEMFILE)).await.unwrap(), + GEMFILE_TRANSITIVE + ); + assert_eq!( + tokio::fs::read_to_string(root.join(GEMFILE_LOCK)) + .await + .unwrap(), + LOCK_TRANSITIVE + ); + } + + /// A re-vendor must RE-SORT: the replacement PATH section lands wherever + /// the NEW uuid sorts among the other vendored gems' sections, not where + /// the old one sat. + #[tokio::test] + async fn test_revendor_new_uuid_resorts_path_sections() { + let (_tmp, root, installed_rack, blobs, record_rack) = + fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; + let (installed_puma, record_puma) = add_puma_fixture(&installed_rack, &blobs).await; + for (purl, installed, record) in [ + (PURL, &installed_rack, &record_rack), + (PURL_PUMA, &installed_puma, &record_puma), + ] { + let (result, _e, _w) = + unwrap_done(run_vendor_purl(purl, &root, &blobs, installed, record, false).await); + assert!(result.success, "vendor {purl} failed: {:?}", result.error); + } + + // The patch update moves rack to a uuid sorting BEFORE puma's. + let mut rack2 = record_rack.clone(); + rack2.uuid = UUID2.to_string(); + let (result, _e, _w) = + unwrap_done(run_vendor(&root, &blobs, &installed_rack, &rack2, false).await); + assert!(result.success, "re-vendor must succeed: {:?}", result.error); + + let lock = tokio::fs::read_to_string(root.join(GEMFILE_LOCK)) + .await + .unwrap(); + assert_eq!( + lock, + expected_lock_two_path() + .replace( + &format!("PATH\n remote: {puma}\n specs:\n puma (6.4.2)\n nio4r (~> 2.0)\n\nPATH\n remote: {rack}\n specs:\n rack (3.2.6)\n base64 (>= 0.1.0)\n\n", puma = puma_rel(), rack = copy_rel()), + &format!("PATH\n remote: {rack}\n specs:\n rack (3.2.6)\n base64 (>= 0.1.0)\n\nPATH\n remote: {puma}\n specs:\n puma (6.4.2)\n nio4r (~> 2.0)\n\n", puma = puma_rel(), rack = copy_rel().replace(UUID, UUID2)), + ), + "rack's section moved to the new uuid's sorted position" + ); + } + + /// On a re-vendor over a CHECKSUMS lock the checksum record must ride + /// AGAIN with `original: None`: dropped, the first run's registry + /// `sha256=` line would vanish from the ledger with the replaced entry, + /// and a post-update revert would leave a bare CHECKSUMS entry on a + /// registry gem (frozen installs exit 16). + #[tokio::test] + async fn test_revendor_new_uuid_checksums_keeps_restore_data() { + let (_tmp, root, installed, blobs, record) = + fixture_318(SPIKE_GEMFILE_CHECKSUMS, SPIKE_LOCK_CHECKSUMS_BEFORE).await; + let (r1, e1, _) = + unwrap_done(run_vendor_318(&root, &blobs, &installed, &record, false).await); + assert!(r1.success); + let entry1 = e1.unwrap(); + + let mut record2 = record.clone(); + record2.uuid = UUID2.to_string(); + let (r2, e2, _) = + unwrap_done(run_vendor_318(&root, &blobs, &installed, &record2, false).await); + assert!(r2.success, "re-vendor must succeed: {:?}", r2.error); + assert_eq!( + tokio::fs::read_to_string(root.join(GEMFILE_LOCK)) + .await + .unwrap(), + expected_lock_checksums().replace(UUID, UUID2) + ); + + let mut entry2 = e2.unwrap(); + assert_eq!(entry2.wiring.len(), 3, "{:?}", entry2.wiring); + let ck = &entry2.wiring[2]; + assert_eq!(ck.kind, LOCK_CHECKSUM_WIRING_KIND); + assert!(ck.original.is_none(), "{:?}", ck.original); + assert_eq!( + ck.new.as_ref().unwrap(), + &Value::String(" rack (3.1.8)".to_string()) + ); + + carry_forward_originals(&entry1, &mut entry2); + let outcome = revert_gem(&entry2, &root, false).await; + assert!(outcome.success, "{:?}", outcome.error); + assert!( + !outcome + .warnings + .iter() + .any(|w| w.code == "vendor_lock_entry_drifted"), + "clean revert must not report drift: {:?}", + outcome.warnings + ); + assert_eq!( + tokio::fs::read_to_string(root.join(GEMFILE_LOCK)) + .await + .unwrap(), + SPIKE_LOCK_CHECKSUMS_BEFORE, + "registry sha256 line restored" + ); + assert_eq!( + tokio::fs::read_to_string(root.join(GEMFILE)).await.unwrap(), + SPIKE_GEMFILE_CHECKSUMS + ); + } + + /// The GEM-specs twin of `test_checksums_platform_sibling_fails_closed`: + /// on a bundler < 2.6 lock (no CHECKSUMS section to catch it in) a + /// platform-suffixed sibling spec must fail the lift closed — lifting + /// only the plain entry would leave the sibling behind as a stale + /// registry spec. + #[test] + fn test_gem_specs_platform_sibling_fails_closed() { + let lock = "GEM\n remote: https://rubygems.org/\n specs:\n nokogiri (1.16.0)\n racc (~> 1.4)\n nokogiri (1.16.0-arm64-darwin)\n racc (~> 1.4)\n\nPLATFORMS\n arm64-darwin\n ruby\n\nDEPENDENCIES\n nokogiri\n\nBUNDLED WITH\n 2.5.22\n"; + let rel = format!(".socket/vendor/gem/{UUID}/nokogiri-1.16.0"); + let err = match edit_lock(lock, "nokogiri", "1.16.0", &rel) { + Err(e) => e, + Ok(_) => panic!("a platform-suffixed GEM specs sibling must fail closed"), + }; + assert!(err.contains("platform-suffixed"), "{err}"); + } + /// Trailing options on the declaration (`require: false`, `group: :test`, /// …) must survive the rewrite: dropping `require: false` auto-requires /// the gem at boot, changing app behavior while vendored (the redirect diff --git a/crates/socket-patch-core/src/vendor/registry_fetch.rs b/crates/socket-patch-core/src/vendor/registry_fetch.rs index 0f0c92a5..532f02f2 100644 --- a/crates/socket-patch-core/src/vendor/registry_fetch.rs +++ b/crates/socket-patch-core/src/vendor/registry_fetch.rs @@ -26,6 +26,7 @@ use sha2::{Digest, Sha256, Sha384, Sha512}; use crate::constants::USER_AGENT; use crate::crawlers::go_crawler::encode_module_path; use crate::patch::apply::is_safe_relative_subpath; +use crate::patch::path_safety::is_safe_single_segment; use super::lock_inventory::{LockIntegrity, LockfileEntry}; @@ -237,6 +238,18 @@ async fn fetch_gem( entry: &LockfileEntry, client: &reqwest::Client, ) -> Result { + // The staged leaf must be the canonical `{name}-{version}`: the gem + // vendor backend refuses any other leaf as a platform-suffixed install + // (`platform_gem_unsupported`), so a generic name would kill the whole + // auto-fetch path. The coordinates thereby become a tempdir path + // component — `inventory_gemfile_lock` already filters both, but + // re-assert locally (defense in depth), before any network I/O. + if !is_safe_single_segment(&entry.name) || !is_safe_single_segment(&entry.version) { + return Err(FetchError::Failed(format!( + "unsafe gem coordinates `{}` @ `{}` — refusing to stage", + entry.name, entry.version + ))); + } let Some(url) = entry.resolved.clone() else { return Err(FetchError::Unverifiable(format!( "no download URL for {}@{}", @@ -248,7 +261,7 @@ async fn fetch_gem( let tmp = tempfile::tempdir() .map_err(|e| FetchError::Failed(format!("cannot create fetch tempdir: {e}")))?; - let dir = tmp.path().join("gem"); + let dir = tmp.path().join(format!("{}-{}", entry.name, entry.version)); extract_gem_data(&bytes, &dir).map_err(FetchError::Failed)?; Ok(FetchedPackage { dir, @@ -1482,6 +1495,36 @@ mod tests { "data.tar.gz content extracts at the root (no strip)" ); assert!(fetched.dir().join("README.md").is_file()); + // The staged leaf must be the canonical `{name}-{version}`: + // vendor_gem's platform-suffix guard refuses any other leaf + // (`platform_gem_unsupported`), which killed lockfile auto-fetch + // when this dir was named `gem`. + assert_eq!( + fetched.dir().file_name().unwrap().to_string_lossy(), + "rails-7.1.0", + "staged dir leaf must satisfy vendor_gem's `{{name}}-{{version}}` check" + ); + } + + #[tokio::test] + async fn gem_fetch_refuses_unsafe_coordinates_without_network() { + // The coordinates become the staged-dir leaf, so a separator-bearing + // name must refuse — and BEFORE any I/O (the URL would hard-fail if + // contacted). + let entry = LockfileEntry { + ecosystem: "gem", + name: "ra/ils".into(), + version: "7.1.0".into(), + purl: "pkg:gem/ra/ils@7.1.0".into(), + resolved: Some("http://127.0.0.1:1/nope.gem".into()), + integrity: LockIntegrity::Sha256Hex("0".repeat(64)), + }; + match fetch_and_stage(&entry, &build_registry_client()).await { + Err(FetchError::Failed(msg)) => { + assert!(msg.contains("unsafe gem coordinates"), "{msg}") + } + other => panic!("expected coordinate refusal, got {other:?}"), + } } #[tokio::test] diff --git a/crates/socket-patch-core/tests/crawler_ruby_e2e.rs b/crates/socket-patch-core/tests/crawler_ruby_e2e.rs index ac137863..d611dded 100644 --- a/crates/socket-patch-core/tests/crawler_ruby_e2e.rs +++ b/crates/socket-patch-core/tests/crawler_ruby_e2e.rs @@ -290,6 +290,47 @@ async fn get_gem_paths_vendor_bundle_takes_precedence_over_global() { ); } +/// A JRuby deployment (`bundle install --deployment` under JRuby) puts +/// gems in `vendor/bundle/jruby//gems` — Bundler scopes the path +/// by `Gem.ruby_engine`, not the literal `ruby`. The crawler must discover +/// that layout end-to-end; hardcoding the `ruby` engine dir made JRuby and +/// TruffleRuby deployments silently yield zero gems. No Gemfile is staged, +/// so a regression cannot green via the `gem env` fallback. +#[tokio::test] +#[serial_test::parallel] +async fn get_gem_paths_vendor_bundle_jruby_engine_layout() { + let tmp = tempfile::tempdir().unwrap(); + let jruby_gems = tmp + .path() + .join("vendor") + .join("bundle") + .join("jruby") + .join("3.1.4.0") + .join("gems"); + tokio::fs::create_dir_all(jruby_gems.join("rails-7.1.0").join("lib")) + .await + .unwrap(); + + let crawler = RubyCrawler; + let paths = crawler + .get_gem_paths(&options_at(tmp.path())) + .await + .unwrap(); + assert_eq!( + paths, + vec![jruby_gems.clone()], + "vendor/bundle/jruby/*/gems must be discovered; got {paths:?}" + ); + + // End-to-end: the gem inside the jruby scope is crawled. + let crawled = crawler.crawl_all(&options_at(tmp.path())).await; + let purls: Vec<&str> = crawled.iter().map(|p| p.purl.as_str()).collect(); + assert!( + purls.contains(&"pkg:gem/rails@7.1.0"), + "gem in a jruby vendor scope must be crawled; got {purls:?}" + ); +} + #[tokio::test] #[serial_test::parallel] async fn get_gem_paths_no_gemfile_returns_empty() { diff --git a/crates/socket-patch-core/tests/fixtures/redirect/gem/bundler/basic/expected-edits.json b/crates/socket-patch-core/tests/fixtures/redirect/gem/bundler/basic/expected-edits.json index 9467dc3d..314c0958 100644 --- a/crates/socket-patch-core/tests/fixtures/redirect/gem/bundler/basic/expected-edits.json +++ b/crates/socket-patch-core/tests/fixtures/redirect/gem/bundler/basic/expected-edits.json @@ -12,6 +12,7 @@ "kind": "redirect_gemfile_lock_checksum", "action": "rewritten", "key": "rails", + "original": "rails (7.0.0) sha256=2222222222222222222222222222222222222222222222222222222222222222", "new": "rails (7.0.0) sha256=deadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeefdeadbeef" } ] From 99f9cd0cd476de62eec33a4de55b3752d9cd2d35 Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Thu, 13 Aug 2026 17:15:32 -0700 Subject: [PATCH 2/4] fix(review): CRLF-tolerant source-block guard; dry-run VEX skip for all scan arms MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Addresses both Bugbot findings on #175: - The grant-agnostic Gemfile idempotency guard required LF (`do\n`), so a core.autocrlf checkout of a previously-redirected Gemfile was not recognized and the indented gem line inside the block got wrapped in a second, nested source block on re-run. The recognizer now accepts `do\r?\n`; pinned by gemfile_rerun_on_crlf_checkout_never_nests (verified red without the fix). - The dry-run VEX skip only covered the vendor JSON arm; the interactive `scan --vendor --dry-run --vex` path (embed_vex_human) and the JSON `scan --apply --dry-run --vex` path (embed_vex_into_json at the apply fall-through) still generated the document — exiting 1 on a not-yet-vendored project or writing the attestation during --dry-run. The guard now lives at the top of both embed helpers, covering every scan arm; pinned by scan_vendor_dry_run_with_vex_interactive_* and scan_apply_json_dry_run_with_vex_* (both verified red without it). Co-Authored-By: Claude Fable 5 --- .../socket-patch-cli/src/commands/scan/mod.rs | 16 ++++ .../socket-patch-cli/tests/in_process_scan.rs | 86 +++++++++++++++++++ .../src/patch/redirect/mod.rs | 53 +++++++++++- 3 files changed, 154 insertions(+), 1 deletion(-) diff --git a/crates/socket-patch-cli/src/commands/scan/mod.rs b/crates/socket-patch-cli/src/commands/scan/mod.rs index eefed677..e4a5320a 100644 --- a/crates/socket-patch-cli/src/commands/scan/mod.rs +++ b/crates/socket-patch-cli/src/commands/scan/mod.rs @@ -280,6 +280,15 @@ async fn embed_vex_into_json( if vex_args.vex.is_none() || base_code != 0 { return base_code; } + // A dry run is a non-mutating preview: generating here would verify the + // deliberately untouched tree (failing outright on a not-yet-vendored + // project) and write an attestation file to disk. The marker keeps the + // request visible to JSON consumers instead of silently dropping it + // (same shape as the vendor JSON arm's early return). + if common.dry_run { + result["vex"] = serde_json::json!({ "skipped": true, "reason": "dry_run" }); + return base_code; + } let params = vex_args.to_build_params(); match generate_vex_from_manifest_path(common, ¶ms, manifest_path).await { Ok(summary) => { @@ -314,6 +323,13 @@ async fn embed_vex_human( if vex_args.vex.is_none() || base_code != 0 { return base_code; } + // Dry-run twin of the JSON guard above: no generation, no file write. + if common.dry_run { + if !common.silent { + println!("[dry-run] VEX generation skipped. No attestation written."); + } + return base_code; + } let params = vex_args.to_build_params(); match generate_vex_from_manifest_path(common, ¶ms, manifest_path).await { Ok(summary) => { diff --git a/crates/socket-patch-cli/tests/in_process_scan.rs b/crates/socket-patch-cli/tests/in_process_scan.rs index 96cb0111..c0ea588f 100644 --- a/crates/socket-patch-cli/tests/in_process_scan.rs +++ b/crates/socket-patch-cli/tests/in_process_scan.rs @@ -1540,3 +1540,89 @@ async fn scan_vendor_dry_run_with_vex_does_not_write_attestation_file() { "vendor dry run must leave the manifest byte-for-byte unchanged" ); } + +/// The INTERACTIVE arm's twin: `scan --vendor --dry-run --vex` without +/// `--json` returns through `embed_vex_human`, which generated (and wrote) +/// the document for real — exit 1 on a not-yet-vendored project, an +/// attestation file on disk otherwise. The dry-run guard lives in the embed +/// helpers, so both output modes must skip. +#[tokio::test] +#[serial] +async fn scan_vendor_dry_run_with_vex_interactive_does_not_fail_or_write() { + let server = MockServer::start().await; + mock_batch_one(&server).await; + mock_by_package(&server).await; + + let tmp = tempfile::tempdir().unwrap(); + write_root_package_json(tmp.path()); + write_npm_package(tmp.path(), "in-proc-scan", "1.0.0"); + + let vex_path = tmp.path().join("vendor-dry-interactive.vex.json"); + let mut args = default_args(tmp.path()); + args.common.api_url = Some(server.uri()); + args.common.json = false; + args.vendor = true; + args.common.dry_run = true; + args.vex.vex = Some(vex_path.clone()); + + let code = run_scrubbed(args).await; + assert_eq!( + code, 0, + "an interactive vendor dry run must not fail for not-yet-vendored state" + ); + assert!( + !vex_path.exists(), + "interactive dry run must not write the VEX file" + ); +} + +/// `scan --apply --json --dry-run --vex`: the JSON apply arm synthesizes its +/// preview and falls through to `embed_vex_into_json` with apply_code 0, so +/// without the guard the dry run generated and wrote the attestation. +#[tokio::test] +#[serial] +async fn scan_apply_json_dry_run_with_vex_does_not_write_attestation() { + let server = MockServer::start().await; + mock_batch_one(&server).await; + mock_by_package(&server).await; + + let tmp = tempfile::tempdir().unwrap(); + write_root_package_json(tmp.path()); + write_npm_package(tmp.path(), "in-proc-scan", "1.0.0"); + + // Attestable manifest (same fixture as the vendor twin above): metadata + // for the statement, `setup.manual: ["npm"]`, `--vex-no-verify` below. + let socket = tmp.path().join(".socket"); + std::fs::create_dir_all(&socket).unwrap(); + std::fs::write( + socket.join("manifest.json"), + r#"{ "patches": { + "pkg:npm/in-proc-scan@1.0.0": { + "uuid": "11111111-1111-4111-8111-111111111111", + "exportedAt": "2024-01-01T00:00:00Z", + "files": {}, + "vulnerabilities": { "GHSA-aaaa-bbbb-cccc": { + "cves": ["CVE-2024-0001"], "summary": "s", + "severity": "HIGH", "description": "d" + }}, + "description": "x", "license": "MIT", "tier": "free" + } + }, "setup": { "manual": ["npm"] } }"#, + ) + .unwrap(); + + let vex_path = tmp.path().join("apply-dry.vex.json"); + let mut args = default_args(tmp.path()); + args.common.api_url = Some(server.uri()); + args.apply = true; + args.common.dry_run = true; + args.vex.vex = Some(vex_path.clone()); + args.vex.vex_no_verify = true; + + let code = run_scrubbed(args).await; + assert_eq!(code, 0, "apply dry run with attestable state must exit 0"); + assert!( + !vex_path.exists(), + "apply dry run must not write the VEX file even when generation would succeed" + ); +} diff --git a/crates/socket-patch-core/src/patch/redirect/mod.rs b/crates/socket-patch-core/src/patch/redirect/mod.rs index 87740fcf..3f337eed 100644 --- a/crates/socket-patch-core/src/patch/redirect/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/mod.rs @@ -1783,10 +1783,14 @@ fn rewrite_gem( url_pat.replace(®ex::escape(&format!("/{rotating}/")), "/[^/\"]+/"); } } + // `\r?\n`: the rewriter emits LF, but a `core.autocrlf` checkout + // rewrites the working tree to CRLF — the guard must still + // recognize the block there, or the indented `gem` line inside + // it falls through to `gem_line_re` and gets wrapped again. let block_re = Regex::new( &(String::from(r#"(?m)^source "("#) + &url_pat - + r#")" do\n gem ["']"# + + r#")" do\r?\n gem ["']"# + ®ex::escape(&dep.name) + r#"["']"#), ) @@ -3888,6 +3892,53 @@ mod tests { ); } + /// A `core.autocrlf` checkout rewrites a previously-redirected Gemfile to + /// CRLF. The block recognizer must still see the Socket source block + /// there: if it misses, the indented `gem` line inside the block matches + /// `gem_line_re` and gets wrapped in a second, nested source block. + #[test] + fn gemfile_rerun_on_crlf_checkout_never_nests() { + fn ov(token: &str) -> DepOverride { + let mut o = gem_override("rails", "7.0.0"); + o.token = token.into(); + if let Some(r) = o.registry_override.as_mut() { + r.index_url = format!("https://patch.test/gem/{token}/uuid/"); + } + o + } + // The block exactly as run 1 writes it, after a CRLF checkout. + let crlf_gemfile = "source \"https://rubygems.org\"\r\n\r\n\ + source \"https://patch.test/gem/tok-one/uuid/\" do\r\n \ + gem \"rails\", \"7.0.0\"\r\nend\r\n"; + let mut files = BTreeMap::new(); + files.insert("Gemfile".to_string(), crlf_gemfile.to_string()); + + // Same grant: recognized in place, a true no-op. + let same = rewrite_registry_redirect(&files, &[ov("tok-one")]); + assert!( + !same.files.contains_key("Gemfile"), + "same-grant re-run on a CRLF checkout must not rewrite the Gemfile: {:?}", + same.files.get("Gemfile") + ); + + // Rotated grant: URL refreshed inside the existing block, never nested. + let rotated = rewrite_registry_redirect(&files, &[ov("tok-two")]); + let out = rotated + .files + .get("Gemfile") + .expect("rotated grant refreshes the URL on a CRLF checkout"); + assert_eq!( + out.matches("source \"https://patch.test/gem/").count(), + 1, + "exactly one Socket source block, never nested: {out}" + ); + assert!(!out.contains("tok-one"), "old grant token gone: {out}"); + assert!( + out.contains("source \"https://patch.test/gem/tok-two/uuid/\" do\r\n"), + "existing CRLF block body left intact: {out}" + ); + } + /// A gem-level source option (`git:` / `path:` / `github:` / `source:`) /// preserved into the Socket source block OVERRIDES it in bundler's DSL, /// leaving the redirect a silent no-op that still gets attested. Fail From 4b1e31716dab8eaca50df5b41fa2b68c53c86cdf Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Thu, 13 Aug 2026 17:43:19 -0700 Subject: [PATCH 3/4] test(gem): hermetic hosted capstone + CHECKSUMS/transitive/scan--vendor coverage; gems.rb + CRLF redirect fixes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes the gem audit's top coverage gaps with hardened, independent-oracle tests, plus the two small redirect fixes the new tests exercise. Redirect fixes (patch/redirect/mod.rs gem section + hosted.rs candidates): - gems.rb/gems.locked support end-to-end: the candidate list now reads the modern pair and the gem rewriter keys on whichever pair bundler reads (gems.rb wins when both spellings exist — verified on bundler 4.0.15; same order as setup::gem). Diverging spellings fail closed with redirect_gem_gemfile_spellings_diverge. Before: a gems.rb project was a silent hosted-mode no-op. - CRLF Gemfile.lock tolerance: the CHECKSUMS matchers accept \r-terminated lines and edits preserve the file's CRLF endings byte-for-byte. Before: a CRLF lock (legal to bundler — verified via bundle check/frozen install) was misdiagnosed as bundler <2.6 (redirect_gem_no_checksums_section). New coverage: - e2e_redirect_gem_build.rs — FULLY HERMETIC hosted gem capstone: authored gems built with real `gem build`, one wiremock playing the upstream compact index, the Socket patch-registry compact index (production's /patch-registry/gem/// shape, real /versions md5s and /info checksums), and the patches API. scan --mode hosted, then a REAL fresh-checkout `bundle install` (host bundler) pulls the patched .gem: bytes match afterHash, the runtime dep installs because /info declares it, require probe + post-install verified VEX. A gems.rb twin pins the modern spelling end-to-end. A deps red-arm pins the compact-index dependency contract the production server currently violates: a deps-less /info breaks the install with bundler's APIResponseMismatchError (the live-CI signature). A KNOWN-LIMITATION canary pins that on a CHECKSUMS lock (bundler >= 4 default) the current rewrite makes the prescribed unfrozen install fail with "mismatched checksums" (empirically verified; the converged-lock fix shape that a frozen install accepts is documented in the test). - docker_e2e_vendor_gem.rs — lockfile_checksums twin (bundler 2.7 in the image, `bundle lock --add-checksums`): vendor swaps the registry sha256= CHECKSUMS line for bundler's bare path-gem form, a frozen --network none install accepts the rewritten lock byte-stably, and revert restores the registry sha256= line VERBATIM (the exit-16 hazard). - e2e_vendor_gem_build.rs — TRANSITIVE-dep capstone: vendoring rack via a rack-test project appends the managed block + sorted `rack (= v)!` DEPENDENCIES pin, a real frozen fresh-checkout install accepts the pair byte-stably, require probe through the dependent, revert round-trip. - in_process_vendor.rs — gem through `scan --vendor` (mock-proxy API, hermetic bundler layout, no ruby needed): JSON-arm end-to-end (pair edit + artifact + stub gemspec + already_vendored rerun), manifest-drop reconcile byte-restore, and the --detached variant (no manifest, embedded record, vendor --revert exit path). Every rewriter behavior change is pinned by unit tests that fail on the base commit (gems.rb pair routing, diverging-spelling fail-closed, CRLF pin-in-place/insert byte-preservation, CRLF rerun no-op); the gems.rb e2e leg fails without the candidate-list entry. Bundler behaviors (gems.rb preference, lockfile naming, compact-index formats, checksum enforcement, CRLF acceptance) were validated against real bundler 4.0.15 and the image's 2.7.2 before being baked into fixtures. Co-Authored-By: Claude Fable 5 --- .../src/commands/scan/hosted.rs | 5 + .../tests/docker_e2e_vendor_gem.rs | 308 ++++- .../tests/e2e_redirect_gem_build.rs | 1032 +++++++++++++++++ .../tests/e2e_vendor_gem_build.rs | 306 +++++ .../tests/in_process_vendor.rs | 337 +++++- .../src/patch/redirect/mod.rs | 261 ++++- 6 files changed, 2211 insertions(+), 38 deletions(-) create mode 100644 crates/socket-patch-cli/tests/e2e_redirect_gem_build.rs diff --git a/crates/socket-patch-cli/src/commands/scan/hosted.rs b/crates/socket-patch-cli/src/commands/scan/hosted.rs index 340bddf3..7aadfa02 100644 --- a/crates/socket-patch-cli/src/commands/scan/hosted.rs +++ b/crates/socket-patch-cli/src/commands/scan/hosted.rs @@ -35,6 +35,11 @@ const REDIRECT_CANDIDATE_FILES: &[&str] = &[ "packages.lock.json", "Gemfile", "Gemfile.lock", + // Bundler's modern manifest spelling — preferred over Gemfile when both + // exist (the gem rewriter picks the pair bundler reads and fails closed + // on diverging spellings). + "gems.rb", + "gems.locked", "pom.xml", // Maven Trusted Checksums files the fail-closed maven rewriter merges into // (read so an existing user config / checksum set is preserved, not diff --git a/crates/socket-patch-cli/tests/docker_e2e_vendor_gem.rs b/crates/socket-patch-cli/tests/docker_e2e_vendor_gem.rs index 9b99ee11..c5a6cad7 100644 --- a/crates/socket-patch-cli/tests/docker_e2e_vendor_gem.rs +++ b/crates/socket-patch-cli/tests/docker_e2e_vendor_gem.rs @@ -29,13 +29,17 @@ //! Gemfile and Gemfile.lock and removes `.socket/vendor` entirely → //! re-vendor succeeds again. //! -//! This suite deliberately runs against a lock WITHOUT a `CHECKSUMS` section -//! (bundler keeps `lockfile_checksums` opt-in, and CHECKSUMS-aware vendoring -//! is a parallel workstream) — stage 1 hard-asserts that precondition. -//! TODO(v2 gem CHECKSUMS): add the lockfile_checksums variant (fixture with -//! `bundle config set --local lockfile_checksums true` before the first -//! lock; expect the vendored entry rewritten to bundler's bare path-gem -//! CHECKSUMS form per spikes/gem-checksums/). +//! The first test runs against a lock WITHOUT a `CHECKSUMS` section (bundler +//! 2.7 keeps `lockfile_checksums` opt-in) — its stage 1 hard-asserts that +//! precondition. The `lockfile_checksums` twin covers the opt-in flavor: the +//! fixture lock gains `CHECKSUMS` via `bundle lock --add-checksums` +//! (supported by the image's bundler 2.7), vendor must rewrite the gem's +//! registry `sha256=` line to bundler's bare path-gem form (per +//! spikes/gem-checksums/ a leftover registry line on a path-sourced gem +//! makes `bundle install` fail — exit 16 under frozen mode), the frozen +//! offline install must accept the rewritten lock byte-stably, and revert +//! must restore the registry `sha256=` line VERBATIM (the recorded original +//! is the only offline path back). #![cfg(feature = "docker-e2e")] @@ -56,8 +60,13 @@ const UUID: &str = "32323232-3232-4232-8232-323232323232"; /// host capstones). const GHSA: &str = "GHSA-vend-gem-real"; -/// Glue the shared bash helpers onto a stage body and pin the uuid + ghsa. -fn render(stage_body: &str) -> String { +/// The lockfile_checksums twin's identifiers — distinct so a leaked path or +/// vulnerability id from one flavor can never satisfy the other's asserts. +const CK_UUID: &str = "34343434-3434-4434-8434-343434343434"; +const CK_GHSA: &str = "GHSA-vend-gem-ck"; + +/// Glue the shared bash helpers onto a stage body and pin a uuid + ghsa. +fn render_with(stage_body: &str, uuid: &str, ghsa: &str) -> String { format!( "{}{}{}{}", bash_prelude(), @@ -65,8 +74,12 @@ fn render(stage_body: &str) -> String { json_assert_fns(), stage_body ) - .replace("__UUID__", UUID) - .replace("__GHSA__", GHSA) + .replace("__UUID__", uuid) + .replace("__GHSA__", ghsa) +} + +fn render(stage_body: &str) -> String { + render_with(stage_body, UUID, GHSA) } /// Stage 1: real bundler fixture (network OK) + staged marker patch + @@ -98,10 +111,10 @@ RACK_VER=$(sed -n 's/^ rack (\([0-9][0-9.]*\))$/\1/p' Gemfile.lock | head -1) [ -n "$RACK_VER" ] || { cat Gemfile.lock >&2; fail "could not read the resolved rack version from Gemfile.lock"; } echo "resolved rack version: $RACK_VER" >&2 -# Precondition this suite is scoped to: NO CHECKSUMS section (bundler >= 2.6 -# keeps lockfile_checksums opt-in; CHECKSUMS-aware vendoring is a parallel -# workstream — see the module doc TODO). -grep -q '^CHECKSUMS' Gemfile.lock && fail "Gemfile.lock unexpectedly has a CHECKSUMS section — this suite requires the default (no-CHECKSUMS) lock" +# Precondition this test is scoped to: NO CHECKSUMS section (bundler 2.7 +# keeps lockfile_checksums opt-in; the CHECKSUMS flavor is the +# lockfile_checksums twin below). +grep -q '^CHECKSUMS' Gemfile.lock && fail "Gemfile.lock unexpectedly has a CHECKSUMS section — this test requires the default (no-CHECKSUMS) lock" RUBY_API=$(ruby -e 'puts Gem.ruby_api_version') || fail "ruby api version probe" GEM_DIR="vendor/bundle/ruby/$RUBY_API/gems/rack-$RACK_VER" @@ -349,6 +362,227 @@ fn assert_vex_attested_from_host(host_dir: &std::path::Path) { ); } +/// Stage 1 of the lockfile_checksums twin: the fixture lock GAINS a +/// CHECKSUMS section (`bundle lock --add-checksums`, real bundler 2.7), the +/// upstream registry `sha256=` line is captured verbatim for the revert +/// oracle, and vendor must rewrite that line to bundler's bare path-gem form +/// while landing the same pair edit as the no-CHECKSUMS flavor. +const STAGE1_CK: &str = r#" +mkdir -p /workspace/proj && cd /workspace/proj +export SOCKET_OFFLINE=1 +export BUNDLE_APP_CONFIG="$PWD/.bundle" + +cat > Gemfile <<'EOF' +source "https://rubygems.org" + +gem "rack", "~> 3.1" +EOF + +bundle config set --local path vendor/bundle || fail "bundle config set --local path" + +# 1. REAL fixture: bundle install + an opt-in CHECKSUMS lock (bundler 2.7 +# does not write one by default — the twin suite pins that default). +bundle install > /tmp/install.log 2>&1 || { cat /tmp/install.log >&2; fail "bundle install (fixture) failed"; } +bundle lock --add-checksums > /tmp/lock.log 2>&1 || { cat /tmp/lock.log >&2; fail "bundle lock --add-checksums failed"; } + +RACK_VER=$(sed -n 's/^ rack (\([0-9][0-9.]*\))$/\1/p' Gemfile.lock | head -1) +[ -n "$RACK_VER" ] || { cat Gemfile.lock >&2; fail "could not read the resolved rack version from Gemfile.lock"; } +echo "resolved rack version: $RACK_VER" >&2 + +grep -q '^CHECKSUMS$' Gemfile.lock || { cat Gemfile.lock >&2; fail "bundle lock --add-checksums did not add a CHECKSUMS section"; } +UPSTREAM_LINE=$(grep -E "^ rack \($RACK_VER\) sha256=[0-9a-f]{64}$" Gemfile.lock) +[ "$(echo "$UPSTREAM_LINE" | wc -l)" -eq 1 ] && [ -n "$UPSTREAM_LINE" ] \ + || { cat Gemfile.lock >&2; fail "expected exactly one registry sha256 CHECKSUMS line for rack"; } + +RUBY_API=$(ruby -e 'puts Gem.ruby_api_version') || fail "ruby api version probe" +GEM_DIR="vendor/bundle/ruby/$RUBY_API/gems/rack-$RACK_VER" +ORIG="$GEM_DIR/lib/rack.rb" +[ -f "$ORIG" ] || { ls -R vendor/bundle/ruby >&2 || true; fail "$ORIG missing after bundle install"; } +grep -q 'SOCKET_PATCH_VENDOR_E2E' "$ORIG" && fail "probe constant already in $ORIG — fixture not pristine" + +# 2. Marker patch on the ACTUAL installed bytes. +cp "$ORIG" /tmp/patched.rb +cat >> /tmp/patched.rb <<'EOF' + +# SOCKET-PATCH-VENDOR-E2E-MARKER +module Rack + SOCKET_PATCH_VENDOR_E2E = "__UUID__" +end +EOF +PURL="pkg:gem/rack@$RACK_VER" +stage_patch "$PURL" "__UUID__" "lib/rack.rb" "$ORIG" /tmp/patched.rb \ + "__GHSA__" "CVE-2024-88888" + +mkdir -p /workspace/snap +cp Gemfile /workspace/snap/Gemfile.prevendor +cp Gemfile.lock /workspace/snap/Gemfile.lock.prevendor +printf '%s\n' "$UPSTREAM_LINE" > /workspace/snap/upstream-checksum-line +echo "$RACK_VER" > /workspace/snap/rack-ver + +# 3. Vendor (fully offline). +socket-patch vendor --json --offline > /tmp/vendor.json 2>/tmp/vendor.err +RC=$?; cat /tmp/vendor.err >&2 +[ "$RC" -eq 0 ] || { cat /tmp/vendor.json >&2; fail "vendor exited $RC (expected 0)"; } +assert_json_field /tmp/vendor.json '"status": "success"' +assert_summary /tmp/vendor.json applied 1 +assert_summary /tmp/vendor.json failed 0 +echo "===VENDOR RUN VERIFIED===" + +# 4. The CHECKSUMS rewrite: the registry sha256= line becomes bundler's bare +# path-gem entry (a leftover registry line on a path-sourced gem fails +# the next install — spikes/gem-checksums/). +grep -qxF " rack ($RACK_VER)" Gemfile.lock \ + || { cat Gemfile.lock >&2; fail "CHECKSUMS entry not rewritten to the bare path-gem form"; } +grep -q "^ rack ($RACK_VER) sha256=" Gemfile.lock \ + && { cat Gemfile.lock >&2; fail "registry sha256 CHECKSUMS line still present after vendor"; } +grep -q '^CHECKSUMS$' Gemfile.lock || { cat Gemfile.lock >&2; fail "CHECKSUMS section lost by the vendor edit"; } +echo "===CHECKSUMS REWRITE VERIFIED===" + +# 5. Same mandatory pair edit as the no-CHECKSUMS flavor. +COPY_REL=".socket/vendor/gem/__UUID__/rack-$RACK_VER" +[ -d "$COPY_REL" ] || fail "vendored gem dir missing at $COPY_REL" +grep -qF "gem \"rack\", \"$RACK_VER\", path: \"$COPY_REL\"" Gemfile \ + || { cat Gemfile >&2; fail "Gemfile line not rewritten to the exact-pin + path: form"; } +grep -qF " remote: $COPY_REL" Gemfile.lock || { cat Gemfile.lock >&2; fail "PATH remote is not the relative vendored path"; } +grep -qF " rack (= $RACK_VER)!" Gemfile.lock || { cat Gemfile.lock >&2; fail "DEPENDENCIES pin ' rack (= $RACK_VER)!' missing"; } +awk '/^PATH$/{p=NR} /^GEM$/{g=NR} END{exit !(p && g && p&2; fail "PATH section must precede GEM"; } +echo "===LOCK WIRING VERIFIED===" + +# 6. Fresh-checkout staging: ONLY the committable files. +rm -rf /workspace/fresh && mkdir -p /workspace/fresh +cp Gemfile Gemfile.lock /workspace/fresh/ +cp -R .socket /workspace/fresh/.socket +cp -R .bundle /workspace/fresh/.bundle +echo "===STAGE1 VERIFIED===" +exit 0 +"#; + +/// Stage 2 of the twin (`--network none` + `BUNDLE_FROZEN=true`): the +/// exit-16 hazard proof — a FROZEN cold-cache offline install must accept +/// the CHECKSUMS lock whose entry for the vendored gem is the bare path-gem +/// form, byte-stably, and the probe constant must load from the vendored +/// path. +const STAGE2_CK: &str = r#" +cd /workspace/fresh +export BUNDLE_APP_CONFIG="$PWD/.bundle" +export BUNDLE_FROZEN=true +RACK_VER=$(cat /workspace/snap/rack-ver) + +[ ! -e vendor ] || fail "fresh checkout already has vendor/ (test bug: uncommittable file copied)" +gem list -i '^rack$' > /dev/null && fail "rack pre-installed in the image gem home — cold-cache premise broken" +grep -q '^CHECKSUMS$' Gemfile.lock || fail "fresh checkout lost the CHECKSUMS section (test bug)" + +LOCK_SHA_BEFORE=$(sha256sum Gemfile.lock | cut -d' ' -f1) +bundle install > /tmp/install.log 2>&1 || { cat /tmp/install.log >&2; fail "frozen cold-cache offline bundle install failed on the CHECKSUMS lock"; } +cat /tmp/install.log >&2 +[ "$LOCK_SHA_BEFORE" = "$(sha256sum Gemfile.lock | cut -d' ' -f1)" ] \ + || fail "bundle install churned the committed CHECKSUMS Gemfile.lock" +echo "===FRESH INSTALL VERIFIED===" + +OUT=$(bundle exec ruby -e ' + require "rack" + abort "probe constant missing after require" unless defined?(Rack::SOCKET_PATCH_VENDOR_E2E) + puts Rack::SOCKET_PATCH_VENDOR_E2E + puts $LOADED_FEATURES.grep(%r{/rack\.rb\z}) +' 2>&1) || { echo "$OUT" >&2; fail "bundle exec runtime probe failed"; } +echo "$OUT" >&2 +echo "$OUT" | grep -qF "__UUID__" || fail "probe constant does not carry the patch uuid" +echo "$OUT" | grep -qF ".socket/vendor/gem/__UUID__/rack-$RACK_VER/lib/rack.rb" \ + || fail "rack was not loaded from the vendored path" +echo "===RUNTIME MARKER VERIFIED===" +exit 0 +"#; + +/// Stage 3 of the twin (`--network none`): idempotent re-vendor → revert +/// restores the registry `sha256=` CHECKSUMS line VERBATIM (byte-identical +/// files) → re-vendor rewrites it back to the bare form. +const STAGE3_CK: &str = r#" +cd /workspace/proj +export SOCKET_OFFLINE=1 +export BUNDLE_APP_CONFIG="$PWD/.bundle" +RACK_VER=$(cat /workspace/snap/rack-ver) +UPSTREAM_LINE=$(cat /workspace/snap/upstream-checksum-line) + +# 1. Idempotency: re-run reports already_vendored, both files byte-stable. +GEMFILE_SHA=$(sha256sum Gemfile | cut -d' ' -f1) +LOCK_SHA=$(sha256sum Gemfile.lock | cut -d' ' -f1) +socket-patch vendor --json --offline > /tmp/revendor.json 2>/tmp/revendor.err +RC=$?; cat /tmp/revendor.err >&2 +[ "$RC" -eq 0 ] || { cat /tmp/revendor.json >&2; fail "re-vendor exited $RC"; } +assert_summary /tmp/revendor.json failed 0 +assert_json_field /tmp/revendor.json '"already_vendored"' +[ "$LOCK_SHA" = "$(sha256sum Gemfile.lock | cut -d' ' -f1)" ] || fail "re-vendor churned Gemfile.lock" +[ "$GEMFILE_SHA" = "$(sha256sum Gemfile | cut -d' ' -f1)" ] || fail "re-vendor churned Gemfile" +echo "===IDEMPOTENT VERIFIED===" + +# 2. Revert: byte-restore, INCLUDING the registry sha256= line verbatim +# (the explicit grep keeps the exit-16 hazard documented even if the +# byte-identity assert is ever loosened). +socket-patch vendor --revert --json --offline > /tmp/revert.json 2>/tmp/revert.err +RC=$?; cat /tmp/revert.err >&2 +[ "$RC" -eq 0 ] || { cat /tmp/revert.json >&2; fail "revert exited $RC"; } +assert_summary /tmp/revert.json removed 1 +cmp -s Gemfile /workspace/snap/Gemfile.prevendor \ + || { diff /workspace/snap/Gemfile.prevendor Gemfile >&2 || true; fail "revert did not byte-restore the Gemfile"; } +cmp -s Gemfile.lock /workspace/snap/Gemfile.lock.prevendor \ + || { diff /workspace/snap/Gemfile.lock.prevendor Gemfile.lock >&2 || true; fail "revert did not byte-restore Gemfile.lock"; } +grep -qxF "$UPSTREAM_LINE" Gemfile.lock \ + || { cat Gemfile.lock >&2; fail "revert did not restore the registry sha256= CHECKSUMS line verbatim"; } +[ ! -e .socket/vendor ] || fail ".socket/vendor must be fully removed after revert" +echo "===REVERT VERIFIED===" + +# 3. Re-vendor after revert: the CHECKSUMS entry goes bare again. +socket-patch vendor --json --offline > /tmp/revendor2.json 2>/tmp/revendor2.err +RC=$?; cat /tmp/revendor2.err >&2 +[ "$RC" -eq 0 ] || { cat /tmp/revendor2.json >&2; fail "post-revert re-vendor exited $RC"; } +assert_summary /tmp/revendor2.json applied 1 +grep -qxF " rack ($RACK_VER)" Gemfile.lock || { cat Gemfile.lock >&2; fail "re-vendor did not re-bare the CHECKSUMS entry"; } +grep -q "^ rack ($RACK_VER) sha256=" Gemfile.lock && { cat Gemfile.lock >&2; fail "registry sha256 line back after re-vendor"; } +echo "===REVENDOR VERIFIED===" +exit 0 +"#; + +/// Host-side oracle for the lockfile_checksums twin: the pair edit is wired +/// AND the CHECKSUMS section holds bundler's bare path-gem entry for rack +/// (no registry `sha256=` remnant) — asserted from the mounted files without +/// trusting the in-container greps. +fn assert_ck_pair_wired_from_host(host_dir: &std::path::Path) { + let rack_ver = std::fs::read_to_string(host_dir.join("snap/rack-ver")) + .expect("snap/rack-ver") + .trim() + .to_string(); + let copy_rel = format!(".socket/vendor/gem/{CK_UUID}/rack-{rack_ver}"); + + let gemfile = + std::fs::read_to_string(host_dir.join("proj/Gemfile")).expect("read mounted Gemfile"); + assert!( + gemfile.contains(&format!( + "gem \"rack\", \"{rack_ver}\", path: \"{copy_rel}\"" + )), + "host oracle: Gemfile not in the exact-pin + path: form:\n{gemfile}" + ); + + let lock = std::fs::read_to_string(host_dir.join("proj/Gemfile.lock")) + .expect("read mounted Gemfile.lock"); + assert!( + lock.contains("\nCHECKSUMS\n"), + "host oracle: this twin must run against a CHECKSUMS lock:\n{lock}" + ); + assert!( + lock.contains(&format!("\n rack ({rack_ver})\n")), + "host oracle: bare path-gem CHECKSUMS entry missing:\n{lock}" + ); + assert!( + !lock.contains(&format!(" rack ({rack_ver}) sha256=")), + "host oracle: registry sha256 CHECKSUMS line survived the vendor edit:\n{lock}" + ); + assert!( + lock.contains(&format!("\n rack (= {rack_ver})!")), + "host oracle: DEPENDENCIES pin missing:\n{lock}" + ); +} + #[test] fn gem_vendor_fresh_checkout_bundle_install_and_revert() { if skip_if_no_image(IMAGE) { @@ -388,3 +622,47 @@ fn gem_vendor_fresh_checkout_bundle_install_and_revert() { // Suite leaves the project re-vendored; the host oracle must hold again. assert_pair_wired_from_host(&host_dir); } + +/// The `lockfile_checksums` twin (see the module doc): same lifecycle against +/// a lock WITH a CHECKSUMS section — the vendor edit must swap the registry +/// `sha256=` line for bundler's bare path-gem entry, the frozen offline +/// install must accept it byte-stably, and revert must restore the registry +/// line verbatim. +#[test] +fn gem_vendor_lockfile_checksums_fresh_checkout_and_revert() { + if skip_if_no_image(IMAGE) { + return; + } + let tmp = tempfile::tempdir().expect("tempdir"); + let host_dir = tmp.path().canonicalize().expect("canonicalize tempdir"); + + // Stage 1 — networked fixture install + --add-checksums + offline vendor + // + CHECKSUMS-rewrite + pair-edit asserts. + let out = run_in_image(IMAGE, &host_dir, &render_with(STAGE1_CK, CK_UUID, CK_GHSA)); + assert_stage_markers( + "gem ck stage 1 (install+add-checksums+vendor)", + &out, + &["VENDOR RUN", "CHECKSUMS REWRITE", "LOCK WIRING", "STAGE1"], + ); + assert_ck_pair_wired_from_host(&host_dir); + + // Stage 2 — fresh checkout, frozen + cold caches + network cut: the + // exit-16 hazard proof on the CHECKSUMS lock. + let out = + run_in_image_network_none(IMAGE, &host_dir, &render_with(STAGE2_CK, CK_UUID, CK_GHSA)); + assert_stage_markers( + "gem ck stage 2 (fresh checkout, --network none, BUNDLE_FROZEN)", + &out, + &["FRESH INSTALL", "RUNTIME MARKER"], + ); + + // Stage 3 — idempotency, revert (verbatim sha256= restore), re-vendor. + let out = + run_in_image_network_none(IMAGE, &host_dir, &render_with(STAGE3_CK, CK_UUID, CK_GHSA)); + assert_stage_markers( + "gem ck stage 3 (idempotent+revert+re-vendor)", + &out, + &["IDEMPOTENT", "REVERT", "REVENDOR"], + ); + assert_ck_pair_wired_from_host(&host_dir); +} diff --git a/crates/socket-patch-cli/tests/e2e_redirect_gem_build.rs b/crates/socket-patch-cli/tests/e2e_redirect_gem_build.rs new file mode 100644 index 00000000..b017f277 --- /dev/null +++ b/crates/socket-patch-cli/tests/e2e_redirect_gem_build.rs @@ -0,0 +1,1032 @@ +//! Real-bundler hosted-mode capstone e2e for gem — the full-chain proof for +//! `scan --mode hosted` on the rubygems-compact-index override, and the +//! executable pin on the compact-index DEPENDENCY contract the production +//! server currently violates (its `/info` answers `{"error":"not_built"}` and +//! the `/api/v1/dependencies` fallback returns a zero-byte body — see +//! `e2e_hosted_production.rs`'s `is_known_defect` tolerance). +//! +//! Unlike the npm/cargo siblings, this suite is FULLY hermetic: the fixture +//! gems are authored here and built with the real `gem build`, and ONE +//! wiremock plays every server in the chain — +//! +//! * the UPSTREAM rubygems registry (compact index `/versions`, +//! `/info/`, `/names`, `/gems/-.gem`) serving +//! `vuln-gem` 1.0.0 (which `require`s its runtime dependency `tiny-dep`) +//! and `tiny-dep` 1.0.0, +//! * the Socket PATCH REGISTRY compact index (same protocol, production's +//! `/patch-registry/gem///` base) serving the PATCHED +//! `vuln-gem` — with `/info` correctly declaring the `tiny-dep` runtime +//! dependency and the patched `.gem`'s sha256 checksum, +//! * the Socket patches API (batch / by-package / package-reference / view). +//! +//! The chain proven against the REAL host bundler: +//! +//! 1. `bundle install` the fixture project from the mock upstream into a +//! project-local `vendor/bundle` (no rubygems.org, no network beyond +//! loopback). +//! 2. `scan --mode hosted --json --vex …` (the real binary): the Gemfile +//! gains the `source "" do … end` block, the ledger embeds +//! the patch record, the in-run VEX is the unverified `(redirected)` +//! attestation. +//! 3. FRESH-CHECKOUT PROOF: only the committable files travel; an UNFROZEN +//! `bundle install` (the flow the rewriter's `redirect_gem_frozen_install` +//! warning prescribes) resolves the patched gem from the mock patch +//! registry: installed bytes byte-match the patch blob, the runtime dep +//! installs BECAUSE the registry `/info` declares it, and a require +//! probe loads the patched code. +//! 4. POST-INSTALL VERIFIED VEX: `socket-patch vex` hash-verifies the +//! installed tree against the ledger record. +//! +//! The `gems.rb` twin drives the same chain through bundler's modern +//! `gems.rb`/`gems.locked` spelling (which bundler prefers over `Gemfile` +//! when both exist — this pins the candidate-list + rewriter support). +//! +//! The deps red-arm serves a PRODUCTION-LIKE `/info` (checksum but NO +//! dependencies): the fresh install must fail with bundler's +//! `APIResponseMismatchError … revealed dependencies not in the API` — the +//! exact live-CI signature — so any server or fixture that stops declaring +//! runtime deps turns this suite red. +//! +//! KNOWN LIMITATION, pinned as a canary: on a lock that carries a CHECKSUMS +//! section (bundler >= 4 writes one by default), today's rewrite (Gemfile +//! block + CHECKSUMS pin, GEM section left on the upstream remote) makes the +//! prescribed unfrozen install fail with "Bundler found mismatched checksums" +//! — bundler still attributes the gem to the upstream source and refuses the +//! lockfile-vs-API disagreement (exit 37, verified on bundler 4.0.15). The +//! canary test pins that reality; the verified fix shape is the fully +//! converged lock (patched-registry GEM section + ` (= )!` +//! DEPENDENCIES pin + patched CHECKSUMS sha — a frozen install of that shape +//! passes), which must land in the TS twin + golden fixtures together. +//! +//! Skips (with a println) when `ruby`/`gem`/`bundle` are missing or the host +//! bundler predates 2.6 (the CHECKSUMS-aware floor); everything after that is +//! hard — no live network is involved at all. + +use std::io::Write as _; +use std::path::{Path, PathBuf}; +use std::process::{Command, Output, Stdio}; + +use sha2::{Digest, Sha256}; +use socket_patch_core::hash::git_sha256::compute_git_sha256_from_bytes; +use wiremock::matchers::{method, path, path_regex}; +use wiremock::{Mock, MockServer, ResponseTemplate}; + +#[path = "common/cache_env.rs"] +mod cache_env; + +const ORG: &str = "test-org"; +const DEP: &str = "vuln-gem"; +const DEP_VERSION: &str = "1.0.0"; +const TRANSITIVE: &str = "tiny-dep"; +/// Canonical lowercase patch uuid — a path level of both the hosted artifact +/// URL and the patch-registry index URL (production shape). +const UUID: &str = "7c8d9e0f-1a2b-4a1b-8c2d-3e4f5a6b7c8d"; +/// Access-token uuid segment of the hosted URLs (opaque to the CLI — it +/// writes what the reference endpoint hands back). +const TOKEN: &str = "44444444-4444-4444-8444-444444444444"; +const GHSA: &str = "GHSA-redirect-gem-real"; +const PRODUCT: &str = "pkg:gem/app@1.0.0"; +const PURL: &str = "pkg:gem/vuln-gem@1.0.0"; + +/// The runtime probe constant baked into the PATCHED lib — observable at +/// `require` time, carries the patch uuid so the assert can't pass on any +/// other content. +fn patched_marker() -> String { + format!("PATCHED-{UUID}") +} + +/// The pristine gem sources. `vuln-gem` REQUIRES its runtime dependency at +/// load time, so a resolution that drops `tiny-dep` (what a deps-less +/// registry `/info` produces) cannot pass the require probe. +fn orig_lib() -> String { + "require \"tiny_dep\"\n\nmodule VulnGem\n def self.status\n \"VULNERABLE\"\n end\n\n DEP = TinyDep::VALUE\nend\n".to_string() +} + +fn patched_lib() -> String { + orig_lib().replace("\"VULNERABLE\"", &format!("\"{}\"", patched_marker())) +} + +const TINY_LIB: &str = "module TinyDep\n VALUE = \"tiny-ok\"\nend\n"; + +// ── self-contained helpers ──────────────────────────────────────────── + +fn binary() -> PathBuf { + PathBuf::from(env!("CARGO_BIN_EXE_socket-patch")) +} + +fn has_command(cmd: &str) -> bool { + let mut probe = Command::new(cmd); + probe.arg("--version"); + cache_env::isolate(&mut probe); + probe + .stdout(Stdio::null()) + .stderr(Stdio::null()) + .status() + .is_ok() +} + +/// `bundle --version` → `(major, minor)`; `None` = no usable bundler. +fn bundler_version() -> Option<(u32, u32)> { + let mut probe = Command::new("bundle"); + probe.arg("--version"); + cache_env::isolate(&mut probe); + let out = probe.output().ok()?; + if !out.status.success() { + return None; + } + let text = String::from_utf8_lossy(&out.stdout).trim().to_string(); + let ver = text.split_whitespace().last()?.to_string(); + let mut it = ver.split('.'); + let major = it.next()?.parse().ok()?; + let minor = it.next()?.parse().ok()?; + Some((major, minor)) +} + +/// Run the socket-patch binary with the ambient `SOCKET_*` surface scrubbed +/// (a developer's `SOCKET_DRY_RUN=1` must not steer the assertions) and +/// `VIRTUAL_ENV` (crawler discovery input) removed. +fn run_socket(cwd: &Path, args: &[&str]) -> (i32, String, String) { + let mut cmd = Command::new(binary()); + cmd.args(args).current_dir(cwd); + for (k, _) in std::env::vars_os() { + if k.to_string_lossy().starts_with("SOCKET_") && k.to_string_lossy() != "SOCKET_NO_CONFIG" { + cmd.env_remove(&k); + } + } + cmd.env_remove("VIRTUAL_ENV"); + let out = cmd.output().expect("failed to run socket-patch binary"); + ( + out.status.code().unwrap_or(-1), + String::from_utf8_lossy(&out.stdout).into_owned(), + String::from_utf8_lossy(&out.stderr).into_owned(), + ) +} + +/// Run `bundle ` in `cwd`: ambient `BUNDLE_*`/`GEM_*` scrubbed, caches +/// isolated, `BUNDLE_APP_CONFIG` pinned to the project's own `.bundle/`, and +/// a PER-PROJECT `BUNDLE_USER_HOME` so each stage's compact-index cache is +/// cold (the fresh-checkout install must be forced through the wiremock +/// registry, never satisfied from the scan project's cache). +fn bundle(cwd: &Path, args: &[&str]) -> Output { + let mut cmd = Command::new("bundle"); + cmd.args(args).current_dir(cwd); + for (k, _) in std::env::vars_os() { + let key = k.to_string_lossy().into_owned(); + if key.starts_with("BUNDLE_") || key.starts_with("GEM_") { + cmd.env_remove(&k); + } + } + cache_env::isolate(&mut cmd); + cmd.env("BUNDLE_APP_CONFIG", cwd.join(".bundle")); + cmd.env("BUNDLE_USER_HOME", cwd.join(".bundle-user-home")); + cmd.output().expect("failed to run bundle") +} + +fn sha256_hex(bytes: &[u8]) -> String { + hex::encode(Sha256::digest(bytes)) +} + +/// MD5 hex digest via the host ruby (`Digest::MD5`) — the compact-index +/// `/versions` line carries the md5 of each `/info/` body and bundler +/// validates it; ruby is already a suite prerequisite, so no md5 dev-dep. +fn md5_hex(bytes: &[u8]) -> String { + let mut child = Command::new("ruby") + .args(["-rdigest", "-e", "print Digest::MD5.hexdigest(STDIN.read)"]) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::null()) + .spawn() + .expect("failed to run ruby for md5"); + child + .stdin + .take() + .expect("ruby stdin") + .write_all(bytes) + .expect("write md5 input"); + let out = child.wait_with_output().expect("ruby md5 output"); + assert!(out.status.success(), "ruby md5 helper failed"); + let hexstr = String::from_utf8(out.stdout).expect("md5 hex is ascii"); + assert_eq!( + hexstr.len(), + 32, + "md5 hex digest must be 32 chars: {hexstr}" + ); + hexstr +} + +fn copy_dir_recursive(src: &Path, dst: &Path) { + std::fs::create_dir_all(dst).unwrap(); + for entry in std::fs::read_dir(src).unwrap() { + let entry = entry.unwrap(); + let to = dst.join(entry.file_name()); + if entry.file_type().unwrap().is_dir() { + copy_dir_recursive(&entry.path(), &to); + } else { + std::fs::copy(entry.path(), &to).unwrap(); + } + } +} + +/// Author a gem (gemspec + one lib file) and build it with the REAL +/// `gem build`; returns the `.gem` bytes. +fn build_gem( + stage: &Path, + name: &str, + version: &str, + lib_file: &str, + lib_content: &str, + runtime_deps: &[&str], +) -> Vec { + let dir = stage.join(format!("{name}-src")); + std::fs::create_dir_all(dir.join("lib")).unwrap(); + std::fs::write(dir.join("lib").join(lib_file), lib_content).unwrap(); + let deps: String = runtime_deps + .iter() + .map(|d| format!(" s.add_dependency \"{d}\", \">= 0\"\n")) + .collect(); + std::fs::write( + dir.join(format!("{name}.gemspec")), + format!( + "Gem::Specification.new do |s|\n s.name = \"{name}\"\n s.version = \"{version}\"\n s.summary = \"socket-patch hosted-gem capstone fixture\"\n s.authors = [\"socket-patch e2e\"]\n s.files = [\"lib/{lib_file}\"]\n s.require_paths = [\"lib\"]\n{deps}end\n" + ), + ) + .unwrap(); + let mut cmd = Command::new("gem"); + cmd.args(["build", &format!("{name}.gemspec")]) + .current_dir(&dir); + cache_env::isolate(&mut cmd); + let out = cmd.output().expect("failed to run gem build"); + assert!( + out.status.success(), + "gem build {name} failed:\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&out.stdout), + String::from_utf8_lossy(&out.stderr), + ); + std::fs::read(dir.join(format!("{name}-{version}.gem"))).expect("built .gem present") +} + +/// One gem a compact index serves: coordinates, runtime deps (compact-index +/// `name:constraint` tokens), and the `.gem` bytes the download route returns. +struct IndexGem { + name: &'static str, + version: &'static str, + deps: Vec, + gem: Vec, +} + +/// Mount a complete rubygems compact index under `base` (no trailing slash): +/// `/versions` (with real per-info md5 digests — bundler validates them), +/// `/info/` (deps + `checksum:`), `/names`, and the +/// `/gems/-.gem` download routes. +async fn mount_compact_index(server: &MockServer, base: &str, gems: &[IndexGem]) { + let mut versions_body = String::from("created_at: 2026-01-01T00:00:00Z\n---\n"); + let mut names_body = String::from("---\n"); + for g in gems { + let deps = g.deps.join(","); + let info_body = format!( + "---\n{} {deps}|checksum:{}\n", + g.version, + sha256_hex(&g.gem) + ); + versions_body.push_str(&format!( + "{} {} {}\n", + g.name, + g.version, + md5_hex(info_body.as_bytes()) + )); + names_body.push_str(&format!("{}\n", g.name)); + Mock::given(method("GET")) + .and(path(format!("{base}/info/{}", g.name))) + .respond_with(ResponseTemplate::new(200).set_body_raw(info_body, "text/plain")) + .mount(server) + .await; + Mock::given(method("GET")) + .and(path(format!("{base}/gems/{}-{}.gem", g.name, g.version))) + .respond_with( + ResponseTemplate::new(200).set_body_raw(g.gem.clone(), "application/octet-stream"), + ) + .mount(server) + .await; + } + Mock::given(method("GET")) + .and(path(format!("{base}/versions"))) + .respond_with(ResponseTemplate::new(200).set_body_raw(versions_body, "text/plain")) + .mount(server) + .await; + Mock::given(method("GET")) + .and(path(format!("{base}/names"))) + .respond_with(ResponseTemplate::new(200).set_body_raw(names_body, "text/plain")) + .mount(server) + .await; +} + +/// Everything the post-redirect legs need. `_server` keeps every registry and +/// API route alive through the fresh `bundle install`. +struct RedirectFixture { + tmp: tempfile::TempDir, + proj: PathBuf, + index_url: String, + gemfile_name: &'static str, + lock_name: &'static str, + patched: Vec, + _server: MockServer, +} + +/// Which manifest spelling the fixture project uses. +#[derive(Clone, Copy, PartialEq)] +enum Spelling { + Gemfile, + GemsRb, +} + +impl Spelling { + fn pair(self) -> (&'static str, &'static str) { + match self { + Spelling::Gemfile => ("Gemfile", "Gemfile.lock"), + Spelling::GemsRb => ("gems.rb", "gems.locked"), + } + } +} + +/// Build the hermetic fixture and run `scan --mode hosted` through the real +/// binary: author + `gem build` the three gems, mount both compact indexes +/// and the patches API, `bundle install` from the mock upstream, scan, and +/// assert the redirect envelope + Gemfile rewrite. `checksums_lock` opts the +/// fixture lock into a CHECKSUMS section (`bundle lock --add-checksums`); +/// `registry_declares_deps` toggles the patch registry's `/info` between the +/// CORRECT contract (runtime deps declared) and today's production-like +/// deps-less answer. `None` = skip (message already printed). +async fn redirect_scanned_project( + tag: &str, + spelling: Spelling, + checksums_lock: bool, + registry_declares_deps: bool, +) -> Option { + for cmd in ["ruby", "gem", "bundle"] { + if !has_command(cmd) { + println!("SKIP e2e_redirect_gem_build ({tag}): `{cmd}` not installed"); + return None; + } + } + let Some((major, minor)) = bundler_version() else { + println!("SKIP e2e_redirect_gem_build ({tag}): `bundle --version` unparseable"); + return None; + }; + // 2.6 floor: the suite exercises CHECKSUMS-aware behavior (lock pins, + // `bundle lock --add-checksums`, `lockfile_checksums` config) that + // predates nothing older. + if major < 2 || (major == 2 && minor < 6) { + println!( + "SKIP e2e_redirect_gem_build ({tag}): host bundler {major}.{minor} predates the \ + CHECKSUMS-aware 2.6 floor" + ); + return None; + } + + let tmp = tempfile::tempdir().unwrap(); + let (gemfile_name, lock_name) = spelling.pair(); + + // 1. Author + build the fixture gems with the real toolchain. + let stage = tmp.path().join("gem-stage"); + let tiny_gem = build_gem(&stage, TRANSITIVE, "1.0.0", "tiny_dep.rb", TINY_LIB, &[]); + let vuln_gem = build_gem( + &stage, + DEP, + DEP_VERSION, + "vuln_gem.rb", + &orig_lib(), + &[TRANSITIVE], + ); + let patched_gem = build_gem( + &stage, + DEP, + DEP_VERSION, + "vuln_gem.rb", + &patched_lib(), + &[TRANSITIVE], + ); + let patched_sha = sha256_hex(&patched_gem); + + // 2. One wiremock plays upstream registry, patch registry, and the API. + let server = MockServer::start().await; + mount_compact_index( + &server, + "/upstream", + &[ + IndexGem { + name: TRANSITIVE, + version: "1.0.0", + deps: vec![], + gem: tiny_gem, + }, + IndexGem { + name: DEP, + version: DEP_VERSION, + deps: vec![format!("{TRANSITIVE}:>= 0")], + gem: vuln_gem, + }, + ], + ) + .await; + // The patch registry: production's `/patch-registry/gem///` + // base. The deps red-arm serves the checksum but NO runtime deps — the + // shape a server that ignores the gem's own gemspec dependencies emits. + let registry_base = format!("/patch-registry/gem/{TOKEN}/{UUID}"); + let index_url = format!("{}{registry_base}/", server.uri()); + mount_compact_index( + &server, + ®istry_base, + &[IndexGem { + name: DEP, + version: DEP_VERSION, + deps: if registry_declares_deps { + vec![format!("{TRANSITIVE}:>= 0")] + } else { + vec![] + }, + gem: patched_gem, + }], + ) + .await; + + let orig = orig_lib().into_bytes(); + let patched = patched_lib().into_bytes(); + let hosted_url = format!( + "{}/patch/gem/{DEP}/{DEP_VERSION}/{TOKEN}/{UUID}/{DEP}-{DEP_VERSION}.gem", + server.uri() + ); + // Batch discovery: the crawled gem has one free patch. + Mock::given(method("POST")) + .and(path(format!("/v0/orgs/{ORG}/patches/batch"))) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ + "packages": [{ + "purl": PURL, + "patches": [{ + "uuid": UUID, "purl": PURL, "tier": "free", + "cveIds": [], "ghsaIds": [], "severity": "high", + "title": "gem redirect capstone fixture" + }] + }], + "canAccessPaidPatches": false, + }))) + .mount(&server) + .await; + // Per-package search used by the redirect selection. + Mock::given(method("GET")) + .and(path_regex(format!( + "^/v0/orgs/{ORG}/patches/by-package/.+$" + ))) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ + "patches": [{ + "uuid": UUID, "purl": PURL, + "publishedAt": "2026-01-01T00:00:00Z", + "description": "x", "license": "MIT", "tier": "free", + "vulnerabilities": {} + }], + "canAccessPaidPatches": false, + }))) + .mount(&server) + .await; + // Reference endpoint: granted, carrying the rubygems-compact-index + // registry override (the identifier shape the TS reference builder + // emits — name / version / gemChecksumSha256). + Mock::given(method("POST")) + .and(path(format!("/v0/orgs/{ORG}/patches/package"))) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ + "results": { + UUID: { + "status": "granted", + "url": hosted_url, + "purl": PURL, + "artifacts": [{ + "kind": "tarball", + "url": hosted_url, + "integrity": { "sha256": patched_sha } + }], + "registryOverride": { + "kind": "rubygems-compact-index", + "indexUrl": index_url, + "identifiers": { + "name": DEP, + "version": DEP_VERSION, + "gemChecksumSha256": patched_sha, + } + } + } + } + }))) + .mount(&server) + .await; + // View endpoint: the patch record (REAL before/after hashes of the + // authored vs patched lib) the redirect run persists for VEX. + Mock::given(method("GET")) + .and(path(format!("/v0/orgs/{ORG}/patches/view/{UUID}"))) + .respond_with(ResponseTemplate::new(200).set_body_json(serde_json::json!({ + "uuid": UUID, + "purl": PURL, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { + "lib/vuln_gem.rb": { + "beforeHash": compute_git_sha256_from_bytes(&orig), + "afterHash": compute_git_sha256_from_bytes(&patched), + } + }, + "vulnerabilities": { + GHSA: { + "cves": ["CVE-2026-3333"], + "summary": "gem redirect capstone vuln", + "severity": "high", + "description": "d" + } + }, + "description": "x", "license": "MIT", "tier": "free" + }))) + .mount(&server) + .await; + + // 3. The fixture project, installed from the MOCK upstream (hermetic). + let proj = tmp.path().join("proj"); + std::fs::create_dir_all(&proj).unwrap(); + std::fs::write( + proj.join(gemfile_name), + format!("source \"{}/upstream\"\n\ngem \"{DEP}\"\n", server.uri()), + ) + .unwrap(); + let config = bundle( + &proj, + &["config", "set", "--local", "path", "vendor/bundle"], + ); + assert!( + config.status.success(), + "bundle config set --local path failed:\n{}", + String::from_utf8_lossy(&config.stderr) + ); + if !checksums_lock { + // Pin the bundler-2.x/3.x lock shape (no CHECKSUMS section) even on a + // bundler >= 4 host, which writes CHECKSUMS into fresh locks by default. + let cfg = bundle( + &proj, + &["config", "set", "--local", "lockfile_checksums", "false"], + ); + assert!(cfg.status.success(), "bundle config lockfile_checksums"); + } + let install = bundle(&proj, &["install"]); + assert!( + install.status.success(), + "fixture `bundle install` against the mock upstream failed:\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&install.stdout), + String::from_utf8_lossy(&install.stderr), + ); + if checksums_lock { + // Idempotent on bundler >= 4 (already written), materializes the + // section on 2.6–3.x hosts. + let add = bundle(&proj, &["lock", "--add-checksums"]); + assert!( + add.status.success(), + "bundle lock --add-checksums failed:\n{}", + String::from_utf8_lossy(&add.stderr) + ); + } + let lock_before = std::fs::read_to_string(proj.join(lock_name)) + .unwrap_or_else(|e| panic!("{lock_name} after fixture install: {e}")); + assert_eq!( + lock_before.contains("\nCHECKSUMS\n"), + checksums_lock, + "fixture lock CHECKSUMS presence must match the arm: {lock_before}" + ); + + // Pristine pre-checks (file AND absence of the marker): the post-install + // byte asserts are circular otherwise. + let mut ruby = Command::new("ruby"); + ruby.args(["-e", "puts Gem.ruby_api_version"]); + cache_env::isolate(&mut ruby); + let api = ruby.output().expect("failed to run ruby"); + assert!(api.status.success(), "ruby api version probe failed"); + let api = String::from_utf8_lossy(&api.stdout).trim().to_string(); + let installed_lib = proj + .join("vendor/bundle/ruby") + .join(&api) + .join("gems") + .join(format!("{DEP}-{DEP_VERSION}")) + .join("lib/vuln_gem.rb"); + assert_eq!( + std::fs::read(&installed_lib).expect("installed lib/vuln_gem.rb"), + orig, + "fixture install must extract the authored pristine bytes" + ); + + // 4. scan --mode hosted --vex: the Gemfile rewrite + the in-run + // (unverified) attestation. + let (code, stdout, stderr) = run_socket( + &proj, + &[ + "scan", + "--mode", + "hosted", + "--json", + "--yes", + "--cwd", + proj.to_str().unwrap(), + "--api-url", + &server.uri(), + "--org", + ORG, + "--api-token", + "fake", + "--vex", + "out.vex.json", + "--vex-product", + PRODUCT, + ], + ); + assert_eq!( + code, 0, + "scan --mode hosted failed.\nstdout:\n{stdout}\nstderr:\n{stderr}" + ); + let env: serde_json::Value = serde_json::from_str(&stdout).unwrap_or_else(|e| { + panic!("scan --mode hosted --json output is not JSON: {e}\nstdout:\n{stdout}") + }); + assert_eq!(env["status"], "success", "envelope: {env}"); + assert_eq!(env["redirect"]["mode"], "hosted", "envelope: {env}"); + assert_eq!( + env["redirect"]["redirected"], 1, + "exactly one dep redirected: {env}" + ); + let rewritten: Vec<&str> = env["redirect"]["rewrittenFiles"] + .as_array() + .expect("rewrittenFiles") + .iter() + .filter_map(|v| v.as_str()) + .collect(); + assert!( + rewritten.contains(&gemfile_name), + "the {gemfile_name} rewrite must be reported: {env}" + ); + let warning_codes: Vec<&str> = env["redirect"]["warnings"] + .as_array() + .expect("warnings") + .iter() + .filter_map(|w| w["code"].as_str()) + .collect(); + assert!( + warning_codes.contains(&"redirect_gem_frozen_install"), + "the frozen-install caveat must be surfaced: {env}" + ); + if checksums_lock { + assert!( + rewritten.contains(&lock_name), + "the CHECKSUMS pin must land in {lock_name}: {env}" + ); + } else { + assert!( + warning_codes.contains(&"redirect_gem_no_checksums_section"), + "a no-CHECKSUMS lock cannot be pinned and must say so: {env}" + ); + assert_eq!( + std::fs::read_to_string(proj.join(lock_name)).unwrap(), + lock_before, + "a no-CHECKSUMS lock must be byte-untouched" + ); + } + assert_eq!(env["vex"]["statements"], 1, "vex block: {env}"); + assert_eq!( + env["vex"]["verified"], false, + "in-run hosted VEX is attested from the ledger, not hash-verified: {env}" + ); + + // The Gemfile rewrite: the declaration moved into the source block whose + // URL is the patch-registry compact index. + let gemfile = std::fs::read_to_string(proj.join(gemfile_name)).unwrap(); + assert!( + gemfile.contains(&format!( + "source \"{index_url}\" do\n gem \"{DEP}\", \"{DEP_VERSION}\"\nend" + )), + "{gemfile_name} must gain the patch-registry source block:\n{gemfile}" + ); + if checksums_lock { + let lock = std::fs::read_to_string(proj.join(lock_name)).unwrap(); + assert!( + lock.contains(&format!(" {DEP} ({DEP_VERSION}) sha256={patched_sha}")), + "the lock CHECKSUMS must pin the PATCHED .gem's sha256:\n{lock}" + ); + } + + // Ledger embeds the patch record so a post-install `vex` can verify. + let ledger = std::fs::read_to_string(proj.join(".socket/vendor/redirect-state.json")).unwrap(); + assert!( + ledger.contains("\"records\"") && ledger.contains(GHSA), + "redirect ledger must embed the patch record + vulnerability: {ledger}" + ); + + Some(RedirectFixture { + tmp, + proj, + index_url, + gemfile_name, + lock_name, + patched, + _server: server, + }) +} + +/// New dir holding ONLY what a git checkout would carry — the manifest pair, +/// `.socket/`, `.bundle/` — then the UNFROZEN `bundle install` the rewriter's +/// `redirect_gem_frozen_install` warning prescribes, with a cold per-dir +/// bundler home. Returns the fresh dir and the install output. +fn fresh_checkout_bundle_install(fx: &RedirectFixture) -> (PathBuf, Output) { + let fresh = fx.tmp.path().join("fresh"); + std::fs::create_dir_all(&fresh).unwrap(); + std::fs::copy(fx.proj.join(fx.gemfile_name), fresh.join(fx.gemfile_name)).unwrap(); + std::fs::copy(fx.proj.join(fx.lock_name), fresh.join(fx.lock_name)).unwrap(); + copy_dir_recursive(&fx.proj.join(".socket"), &fresh.join(".socket")); + copy_dir_recursive(&fx.proj.join(".bundle"), &fresh.join(".bundle")); + assert!( + !fresh.join("vendor").exists(), + "fresh checkout must not carry an installed tree (test bug)" + ); + let install = bundle(&fresh, &["install"]); + (fresh, install) +} + +/// The installed gem's lib file under the fresh checkout's vendor/bundle. +fn fresh_installed_lib(fresh: &Path, gem_leaf: &str, lib: &str) -> PathBuf { + let mut ruby = Command::new("ruby"); + ruby.args(["-e", "puts Gem.ruby_api_version"]); + cache_env::isolate(&mut ruby); + let api = ruby.output().expect("failed to run ruby"); + let api = String::from_utf8_lossy(&api.stdout).trim().to_string(); + fresh + .join("vendor/bundle/ruby") + .join(api) + .join("gems") + .join(gem_leaf) + .join("lib") + .join(lib) +} + +/// Assert the full post-install proof: patched bytes on disk, the runtime +/// dependency present (the compact-index deps contract), and the require +/// probe resolving the patched code + the dep from the fresh vendor path. +fn assert_patched_install(fx: &RedirectFixture, fresh: &Path) { + let installed = std::fs::read(fresh_installed_lib( + fresh, + &format!("{DEP}-{DEP_VERSION}"), + "vuln_gem.rb", + )) + .expect("fresh install must land lib/vuln_gem.rb"); + assert_eq!( + installed, fx.patched, + "fresh install must hold the PATCHED bytes, byte-identical to the hosted .gem's lib" + ); + assert_eq!( + compute_git_sha256_from_bytes(&installed), + compute_git_sha256_from_bytes(&fx.patched), + "installed bytes must hash to the patch record's afterHash" + ); + // The deps contract: `tiny-dep` reaches the install ONLY through the + // patch registry's `/info` declaring it (the fresh resolution re-derives + // vuln-gem's dependencies from that answer — production's deps-less + // answer drops it, see the red-arm twin). + assert!( + fresh_installed_lib(fresh, &format!("{TRANSITIVE}-1.0.0"), "tiny_dep.rb").is_file(), + "the runtime dependency must install alongside the patched gem" + ); + let probe = bundle( + fresh, + &[ + "exec", + "ruby", + "-e", + "require \"vuln_gem\"\nputs VulnGem.status\nputs TinyDep::VALUE\nputs $LOADED_FEATURES.grep(%r{/vuln_gem\\.rb\\z})", + ], + ); + assert!( + probe.status.success(), + "bundle exec require probe failed.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&probe.stdout), + String::from_utf8_lossy(&probe.stderr), + ); + let out = String::from_utf8_lossy(&probe.stdout).into_owned(); + assert!( + out.contains(&patched_marker()), + "the patched status marker must be live at require time:\n{out}" + ); + assert!( + out.contains("tiny-ok"), + "the runtime dep's constant must resolve (deps contract):\n{out}" + ); + assert!( + out.contains("/vendor/bundle/"), + "vuln_gem.rb must load from the fresh project-local install:\n{out}" + ); +} + +// ── the capstones ───────────────────────────────────────────────────── + +// multi_thread: the CLI/gem/bundle subprocesses block a worker thread while +// wiremock keeps serving the API + both compact indexes on the others. +#[tokio::test(flavor = "multi_thread")] +#[ignore = "host capstone: shells out to a real ruby/gem/bundler >= 2.6; the unpinned `test` \ + job skips it, an e2e job with a pinned toolchain runs it via --ignored"] +async fn gem_hosted_fresh_checkout_bundle_install_installs_patched_bytes_and_vex_verifies() { + let Some(fx) = redirect_scanned_project("main", Spelling::Gemfile, false, true).await else { + return; + }; + + // FRESH-CHECKOUT PROOF: the unfrozen install the redirect prescribes + // pulls the patched .gem from the hosted compact index. + let (fresh, install) = fresh_checkout_bundle_install(&fx); + assert!( + install.status.success(), + "fresh-checkout `bundle install` must succeed from the patch registry.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&install.stdout), + String::from_utf8_lossy(&install.stderr), + ); + assert_patched_install(&fx, &fresh); + + // The converged lock records the patch registry as the gem's source and + // bundler's own `!` pin — the state a subsequent frozen install accepts. + let lock = std::fs::read_to_string(fresh.join(fx.lock_name)).unwrap(); + assert!( + lock.contains(&format!("remote: {}", fx.index_url)), + "post-install lock must record the patch-registry source:\n{lock}" + ); + assert!( + lock.contains(&format!("{DEP} (= {DEP_VERSION})!")), + "post-install lock must carry bundler's source-pinned dependency:\n{lock}" + ); + + // POST-INSTALL VERIFIED VEX: default verify mode hash-verifies the + // installed tree against the ledger's patch record. + let doc_path = fresh.join("doc.json"); + let (code, stdout, stderr) = run_socket( + &fresh, + &[ + "vex", + "--output", + doc_path.to_str().unwrap(), + "--product", + PRODUCT, + "--cwd", + fresh.to_str().unwrap(), + ], + ); + assert_eq!( + code, 0, + "post-install vex failed.\nstdout:\n{stdout}\nstderr:\n{stderr}" + ); + let doc: serde_json::Value = + serde_json::from_slice(&std::fs::read(&doc_path).unwrap()).unwrap(); + let stmts = doc["statements"].as_array().unwrap(); + assert_eq!( + stmts.len(), + 1, + "exactly the redirected patch must be attested: {doc}" + ); + assert_eq!(stmts[0]["vulnerability"]["name"], GHSA); + assert_eq!(stmts[0]["status"], "not_affected"); + assert_eq!(stmts[0]["products"][0]["subcomponents"][0]["@id"], PURL); + assert_eq!( + stmts[0]["impact_statement"].as_str().unwrap(), + format!("Patched via Socket patch {UUID} (redirected)"), + "the post-install (hash-verified) attestation must carry the (redirected) marker" + ); +} + +/// Bundler's modern `gems.rb`/`gems.locked` spelling, end to end: the +/// candidate list must read the pair, the rewriter must key its edits to it, +/// and the real bundler must install the patched gem from the redirected +/// gems.rb. Fails without the gems.rb support in either layer. +#[tokio::test(flavor = "multi_thread")] +#[ignore = "host capstone: shells out to a real ruby/gem/bundler >= 2.6; the unpinned `test` \ + job skips it, an e2e job with a pinned toolchain runs it via --ignored"] +async fn gem_hosted_gems_rb_spelling_redirects_and_installs() { + let Some(fx) = redirect_scanned_project("gems.rb", Spelling::GemsRb, false, true).await else { + return; + }; + assert!( + !fx.proj.join("Gemfile").exists() && !fx.proj.join("Gemfile.lock").exists(), + "fixture must exercise the modern spelling exclusively (test bug)" + ); + + let (fresh, install) = fresh_checkout_bundle_install(&fx); + assert!( + install.status.success(), + "fresh-checkout `bundle install` from gems.rb must succeed.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&install.stdout), + String::from_utf8_lossy(&install.stderr), + ); + assert_patched_install(&fx, &fresh); + let lock = std::fs::read_to_string(fresh.join("gems.locked")).unwrap(); + assert!( + lock.contains(&format!("remote: {}", fx.index_url)), + "gems.locked must converge on the patch-registry source:\n{lock}" + ); +} + +/// The compact-index DEPENDENCY contract, pinned from the red side: a patch +/// registry whose `/info` omits the gem's runtime deps (today's production +/// behavior — its sidecar index answers `not_built` and the dependency-API +/// fallback is a zero-byte body) BREAKS the prescribed install with +/// bundler's `APIResponseMismatchError`. If the CLI or fixture ever starts +/// tolerating that silently, this turns red. +#[tokio::test(flavor = "multi_thread")] +#[ignore = "host capstone: shells out to a real ruby/gem/bundler >= 2.6; the unpinned `test` \ + job skips it, an e2e job with a pinned toolchain runs it via --ignored"] +async fn gem_hosted_registry_info_without_deps_breaks_install_like_production() { + let Some(fx) = redirect_scanned_project("nodeps", Spelling::Gemfile, false, false).await else { + return; + }; + + let (_fresh, install) = fresh_checkout_bundle_install(&fx); + assert!( + !install.status.success(), + "a deps-less registry /info MUST break the fresh install — a quiet success here means \ + the dependency contract stopped being load-bearing.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&install.stdout), + String::from_utf8_lossy(&install.stderr), + ); + let chatter = format!( + "{}\n{}", + String::from_utf8_lossy(&install.stdout), + String::from_utf8_lossy(&install.stderr) + ); + assert!( + chatter.contains("APIResponseMismatchError") + && chatter.contains("dependencies not in the API"), + "the failure must be bundler's API-mismatch check (the live production signature), \ + not something incidental:\n{chatter}" + ); + // Anti-vacuity: the .gem itself declares the dep, so the mismatch can + // only come from the registry's deps-less /info. + assert!( + chatter.contains(TRANSITIVE), + "the mismatch must name the dropped runtime dep:\n{chatter}" + ); +} + +/// KNOWN-LIMITATION CANARY — CHECKSUMS locks (bundler >= 4 default): the +/// current rewrite (source block + CHECKSUMS pin, GEM section left on the +/// upstream remote) makes the prescribed unfrozen install FAIL: bundler +/// still attributes the gem to the upstream source and refuses the +/// lockfile-vs-upstream-API checksum disagreement ("Bundler found mismatched +/// checksums", exit 37 — verified on bundler 4.0.15). This test pins the +/// rewrite half (the pin lands, its ledger edit records the upstream sha for +/// revert) AND the current install failure. When the rewriter learns the +/// verified fix — the fully converged lock: patched-registry GEM section, +/// ` (= )!` DEPENDENCIES pin, patched CHECKSUMS sha, which a +/// FROZEN install accepts — this canary must flip to asserting success. +#[tokio::test(flavor = "multi_thread")] +#[ignore = "host capstone: shells out to a real ruby/gem/bundler >= 2.6; the unpinned `test` \ + job skips it, an e2e job with a pinned toolchain runs it via --ignored"] +async fn gem_hosted_checksums_lock_pins_patched_sha_but_bundler_refuses_mixed_state() { + let Some(fx) = redirect_scanned_project("checksums", Spelling::Gemfile, true, true).await + else { + return; + }; + + // The rewrite half: the ledger's CHECKSUMS edit must carry the UPSTREAM + // sha as `original` (the only revert path back to the registry line). + let ledger: serde_json::Value = serde_json::from_str( + &std::fs::read_to_string(fx.proj.join(".socket/vendor/redirect-state.json")).unwrap(), + ) + .unwrap(); + let edit = ledger["edits"] + .as_array() + .expect("ledger edits") + .iter() + .find(|e| e["kind"] == "redirect_gemfile_lock_checksum") + .expect("CHECKSUMS pin edit recorded in the ledger"); + assert_eq!(edit["path"], "Gemfile.lock", "edit path: {edit}"); + let original = edit["original"].as_str().expect("original recorded"); + assert!( + original.starts_with(&format!("{DEP} ({DEP_VERSION}) sha256=")), + "original must be the pre-edit registry line: {original}" + ); + assert!( + !std::fs::read_to_string(fx.proj.join("Gemfile.lock")) + .unwrap() + .contains(original), + "the upstream sha line must actually have been replaced (else the pin is vacuous)" + ); + + // The install half — today's reality on a CHECKSUMS lock. + let (_fresh, install) = fresh_checkout_bundle_install(&fx); + assert!( + !install.status.success(), + "KNOWN LIMITATION pinned: if this fresh install now SUCCEEDS, the mixed-state lock \ + handling was fixed — flip this canary to assert success + patched bytes (see the \ + test doc for the verified converged-lock shape).\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&install.stdout), + String::from_utf8_lossy(&install.stderr), + ); + let chatter = format!( + "{}\n{}", + String::from_utf8_lossy(&install.stdout), + String::from_utf8_lossy(&install.stderr) + ); + assert!( + chatter.to_lowercase().contains("mismatched checksums"), + "the refusal must be bundler's checksum-conflict check, not something incidental:\n{chatter}" + ); +} diff --git a/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs b/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs index ad8df879..882d206f 100644 --- a/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs +++ b/crates/socket-patch-cli/tests/e2e_vendor_gem_build.rs @@ -543,3 +543,309 @@ fn gem_vendor_fresh_checkout_bundle_install_and_revert() { ".socket/vendor must be fully removed after revert" ); } + +/// TRANSITIVE-dep capstone: vendoring a gem the Gemfile never declares +/// (`rack`, pulled in by `rack-test`) appends the managed block + the sorted +/// `rack (= )!` DEPENDENCIES pin — a wiring shape the direct-dep +/// capstone never produces — and a REAL frozen `bundle install` of a fresh +/// checkout must accept that pair byte-stably, load the patched bytes from +/// the vendored path through the rack-test require chain, and revert must +/// byte-restore both files (managed block gone, DEPENDENCIES entry deleted). +#[test] +#[ignore = "host capstone: shells out to a real bundler >= 2.5; the unpinned `test` job \ + skips it, the e2e job runs it with a pinned toolchain via --ignored"] +fn gem_vendor_transitive_dep_fresh_checkout_and_revert() { + if !has_command("ruby") { + println!("SKIP e2e_vendor_gem_build (transitive): `ruby` not installed"); + return; + } + let Some((major, minor)) = bundler_version() else { + println!( + "SKIP e2e_vendor_gem_build (transitive): `bundle` not installed (or version \ + unparseable)" + ); + return; + }; + if major < 2 || (major == 2 && minor < 5) { + println!( + "SKIP e2e_vendor_gem_build (transitive): host bundler {major}.{minor} predates \ + the spike-verified 2.5 floor" + ); + return; + } + + let tmp = tempfile::tempdir().unwrap(); + let proj = tmp.path().join("proj"); + std::fs::create_dir_all(&proj).unwrap(); + std::fs::write( + proj.join("Gemfile"), + "source \"https://rubygems.org\"\n\ngem \"rack-test\", \"~> 2.1\"\n", + ) + .unwrap(); + + let config = bundle( + &proj, + &["config", "set", "--local", "path", "vendor/bundle"], + false, + ); + if !config.status.success() { + println!( + "SKIP e2e_vendor_gem_build (transitive): `bundle config set --local path` failed:\n{}", + String::from_utf8_lossy(&config.stderr) + ); + return; + } + // Pin the no-CHECKSUMS lock shape on every host (bundler >= 4 writes a + // CHECKSUMS section by default; 2.5–3.x never do) — the CHECKSUMS-lock + // vendoring flavor is covered by docker_e2e_vendor_gem's twin. + let no_ck = bundle( + &proj, + &["config", "set", "--local", "lockfile_checksums", "false"], + false, + ); + assert!( + no_ck.status.success(), + "bundle config set --local lockfile_checksums failed:\n{}", + String::from_utf8_lossy(&no_ck.stderr) + ); + let install = bundle(&proj, &["install"], false); + if !install.status.success() { + println!( + "SKIP e2e_vendor_gem_build (transitive): `bundle install` failed (registry \ + unreachable, or host ruby too old for rack-test ~> 2.1?):\n{}", + String::from_utf8_lossy(&install.stderr) + ); + return; + } + + let lock_path = proj.join("Gemfile.lock"); + let lock_before = std::fs::read(&lock_path).expect("Gemfile.lock after bundle install"); + let lock_before_text = String::from_utf8_lossy(&lock_before).into_owned(); + let version = locked_gem_version(&lock_before_text, DEP) + .unwrap_or_else(|| panic!("rack-test must resolve rack into Gemfile.lock")); + + // Anti-vacuity: rack really is transitive — undeclared in the Gemfile + // and absent from the lock's DEPENDENCIES section (which does list + // ` rack-test (~> 2.1)`, so the probe pins the exact token). + let gemfile_path = proj.join("Gemfile"); + let gemfile_before = std::fs::read(&gemfile_path).unwrap(); + assert!( + !String::from_utf8_lossy(&gemfile_before).contains("\"rack\""), + "fixture bug: rack must not be Gemfile-declared" + ); + assert!( + !lock_before_text.contains("\nCHECKSUMS\n"), + "fixture bug: this capstone pins the no-CHECKSUMS lock shape: {lock_before_text}" + ); + let deps_section = lock_before_text + .split("DEPENDENCIES\n") + .nth(1) + .and_then(|rest| rest.split("\n\n").next()) + .expect("lock has a DEPENDENCIES section"); + assert!( + !deps_section + .lines() + .any(|l| l == " rack" || l.starts_with(" rack (") || l.starts_with(" rack!")), + "fixture bug: rack must not appear in DEPENDENCIES: {deps_section}" + ); + + // The installed transitive gem, marker patch on its ACTUAL bytes. + let mut ruby = Command::new("ruby"); + ruby.args(["-e", "puts Gem.ruby_api_version"]); + cache_env::isolate(&mut ruby); + let api = ruby.output().expect("failed to run ruby"); + assert!(api.status.success(), "ruby api version probe failed"); + let api = String::from_utf8_lossy(&api.stdout).trim().to_string(); + let installed_rb = proj + .join("vendor/bundle/ruby") + .join(&api) + .join("gems") + .join(format!("{DEP}-{version}")) + .join("lib/rack.rb"); + let orig = std::fs::read(&installed_rb).expect("installed lib/rack.rb"); + assert!( + !String::from_utf8_lossy(&orig).contains("SOCKET_PATCH_VENDOR_E2E"), + "pristine install must not carry the probe constant" + ); + let marker = format!( + "\n# SOCKET-PATCH-VENDOR-E2E-MARKER\nmodule Rack\n SOCKET_PATCH_VENDOR_E2E = \"{UUID}\"\nend\n" + ); + let patched: Vec = [orig.as_slice(), marker.as_bytes()].concat(); + let purl = format!("pkg:gem/{DEP}@{version}"); + stage_patch_with_vuln(&proj, &purl, "lib/rack.rb", &orig, &patched); + + // Vendor (offline). The transitive branch appends the managed block. + let (code, stdout, stderr) = run_socket( + &proj, + &[ + "vendor", + "--json", + "--offline", + "--cwd", + proj.to_str().unwrap(), + ], + ); + assert_eq!( + code, 0, + "vendor failed.\nstdout:\n{stdout}\nstderr:\n{stderr}" + ); + let env = parse_envelope(&stdout); + assert_eq!(env["summary"]["applied"], 1, "one package vendored: {env}"); + assert_eq!(env["summary"]["failed"], 0, "no failures: {env}"); + + // The appended managed block, byte-exact (hand-pinned marker lines — the + // block delimits what revert may delete, so its shape is contract). + let copy_rel = format!(".socket/vendor/gem/{UUID}/{DEP}-{version}"); + let gemfile = std::fs::read_to_string(&gemfile_path).unwrap(); + let expected_gemfile = format!( + "{}# >>> socket-patch vendor (managed) >>>\ngem \"{DEP}\", \"{version}\", \ + path: \"{copy_rel}\"\n# <<< socket-patch vendor (managed) <<<\n", + String::from_utf8_lossy(&gemfile_before) + ); + assert_eq!( + gemfile, expected_gemfile, + "transitive vendor must append exactly the managed block" + ); + + // The lock pair: canonical PATH section before GEM, and the DEPENDENCIES + // pin inserted at bundler's sorted position (rack before rack-test). + let lock = std::fs::read_to_string(&lock_path).unwrap(); + assert!( + lock.contains(&format!( + "PATH\n remote: {copy_rel}\n specs:\n {DEP} ({version})" + )), + "canonical PATH section missing:\n{lock}" + ); + assert!( + lock.contains(&format!( + "DEPENDENCIES\n {DEP} (= {version})!\n rack-test (~> 2.1)\n" + )), + "DEPENDENCIES pin must insert at bundler's sorted position:\n{lock}" + ); + + // FRESH-CHECKOUT PROOF: committable files only, frozen install (bundler + // validates the Gemfile↔lock dependency sets — an unsorted or malformed + // insert fails here), byte-stable lock, patched bytes reached THROUGH the + // rack-test require chain. + let fresh = tmp.path().join("fresh"); + std::fs::create_dir_all(&fresh).unwrap(); + std::fs::copy(&gemfile_path, fresh.join("Gemfile")).unwrap(); + std::fs::copy(&lock_path, fresh.join("Gemfile.lock")).unwrap(); + copy_dir_recursive(&proj.join(".socket"), &fresh.join(".socket")); + copy_dir_recursive(&proj.join(".bundle"), &fresh.join(".bundle")); + assert!( + !fresh.join("vendor").exists(), + "fresh checkout must not carry an installed tree (test bug)" + ); + + let lock_wired = std::fs::read(&lock_path).unwrap(); + let ci = bundle(&fresh, &["install"], true); + assert!( + ci.status.success(), + "fresh-checkout frozen `bundle install` must accept the managed-block pair.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&ci.stdout), + String::from_utf8_lossy(&ci.stderr), + ); + assert_eq!( + std::fs::read(fresh.join("Gemfile.lock")).unwrap(), + lock_wired, + "frozen install must leave the committed Gemfile.lock byte-identical" + ); + + // `require "rack/test"` proves the direct dep still resolves alongside + // the vendored transitive; it does not itself load `lib/rack.rb`, so the + // marker is probed through an explicit `require "rack"` in the same VM. + let probe = bundle( + &fresh, + &[ + "exec", + "ruby", + "-e", + "require \"rack/test\"\n\ + require \"rack\"\n\ + abort \"probe constant missing after require\" unless defined?(Rack::SOCKET_PATCH_VENDOR_E2E)\n\ + puts Rack::SOCKET_PATCH_VENDOR_E2E\n\ + puts $LOADED_FEATURES.grep(%r{/rack\\.rb\\z})", + ], + false, + ); + assert!( + probe.status.success(), + "bundle exec runtime probe failed.\nstdout:\n{}\nstderr:\n{}", + String::from_utf8_lossy(&probe.stdout), + String::from_utf8_lossy(&probe.stderr), + ); + let probe_out = String::from_utf8_lossy(&probe.stdout).into_owned(); + assert!( + probe_out.contains(UUID), + "probe constant must carry the patch uuid:\n{probe_out}" + ); + assert!( + probe_out.contains(&format!("{copy_rel}/lib/rack.rb")), + "rack must be loaded from the vendored path via rack-test:\n{probe_out}" + ); + + // Idempotency: a re-run leaves both files byte-identical (a second + // managed block or a duplicated DEPENDENCIES pin breaks bundler). + let gemfile_wired = std::fs::read(&gemfile_path).unwrap(); + let (code, stdout, stderr) = run_socket( + &proj, + &[ + "vendor", + "--json", + "--offline", + "--cwd", + proj.to_str().unwrap(), + ], + ); + assert_eq!( + code, 0, + "re-vendor failed.\nstdout:\n{stdout}\nstderr:\n{stderr}" + ); + let env2 = parse_envelope(&stdout); + assert_eq!(env2["summary"]["failed"], 0, "re-run must not fail: {env2}"); + assert_eq!( + std::fs::read(&gemfile_path).unwrap(), + gemfile_wired, + "re-vendor must leave the Gemfile byte-identical" + ); + assert_eq!( + std::fs::read(&lock_path).unwrap(), + lock_wired, + "re-vendor must leave Gemfile.lock byte-identical" + ); + + // REVERT PROOF: the managed block and the DEPENDENCIES pin are deletions + // (no pre-vendor original exists for either) — both files must come back + // byte-identical to the pre-vendor snapshots. + let (code, stdout, stderr) = run_socket( + &proj, + &[ + "vendor", + "--revert", + "--json", + "--cwd", + proj.to_str().unwrap(), + ], + ); + assert_eq!( + code, 0, + "revert failed.\nstdout:\n{stdout}\nstderr:\n{stderr}" + ); + let renv = parse_envelope(&stdout); + assert_eq!(renv["summary"]["removed"], 1, "one entry reverted: {renv}"); + assert_eq!( + std::fs::read(&gemfile_path).unwrap(), + gemfile_before, + "revert must restore the Gemfile byte-identical (managed block gone)" + ); + assert_eq!( + std::fs::read(&lock_path).unwrap(), + lock_before, + "revert must restore Gemfile.lock byte-identical (PATH + pin gone)" + ); + assert!( + !proj.join(".socket/vendor").exists(), + ".socket/vendor must be fully removed after revert" + ); +} diff --git a/crates/socket-patch-cli/tests/in_process_vendor.rs b/crates/socket-patch-cli/tests/in_process_vendor.rs index 2b263cce..2c35aa27 100644 --- a/crates/socket-patch-cli/tests/in_process_vendor.rs +++ b/crates/socket-patch-cli/tests/in_process_vendor.rs @@ -1,5 +1,7 @@ //! In-process + envelope contract tests for `socket-patch vendor` (npm -//! backend, plus the golang apply-yields-to-vendor handshake). +//! backend, plus the golang apply-yields-to-vendor handshake, plus the gem +//! backend's `scan --vendor` arm — the one route into the vendor engine no +//! gem project had ever been driven through). //! //! The lifecycle tests call `socket_patch_cli::commands::vendor::run(args)` //! directly (the in-process convention of `in_process_cargo_apply.rs` / @@ -1489,3 +1491,336 @@ async fn offline_service_mode_refuses_instead_of_building() { ); assert_eq!(fx.lock_bytes(), fx.original_lock, "lock untouched"); } + +// ───────────────────────────────────────────────────────────────────── +// 13. gem through `scan --vendor` (mock-proxy API, hermetic bundler layout) +// ───────────────────────────────────────────────────────────────────── + +const GEM_UUID: &str = "35353535-3535-4335-8335-353535353535"; +const GEM_PURL: &str = "pkg:gem/demo-gem@1.0.0"; +const GEM_ORIG: &[u8] = b"module DemoGem\n STATUS = \"orig\"\nend\n"; +const GEM_PATCHED: &[u8] = b"module DemoGem\n STATUS = \"patched\"\nend\n"; +const GEM_GEMSPEC: &str = "Gem::Specification.new do |s|\n s.name = \"demo-gem\"\n s.version = \"1.0.0\"\n s.summary = \"in-process scan --vendor fixture\"\n s.require_paths = [\"lib\"]\nend\n"; +const GEM_GEMFILE: &str = "source \"https://rubygems.org\"\n\ngem \"demo-gem\", \"~> 1.0\"\n"; +/// Hand-pinned bundler lock grammar (no CHECKSUMS — the 2.x/3.x default). +const GEM_LOCK: &str = "GEM\n remote: https://rubygems.org/\n specs:\n demo-gem (1.0.0)\n\nPLATFORMS\n ruby\n\nDEPENDENCIES\n demo-gem (~> 1.0)\n\nBUNDLED WITH\n 2.6.2\n"; + +/// A vendorable gem project in bundler's deployment layout — no real ruby +/// needed: the crawler discovers `vendor/bundle///gems/` under a +/// project with a Gemfile, and the vendor backend reads the stub gemspec from +/// the sibling `specifications/` dir. +struct GemFixture { + tmp: tempfile::TempDir, +} + +impl GemFixture { + fn root(&self) -> &Path { + self.tmp.path() + } + fn gemfile_path(&self) -> PathBuf { + self.root().join("Gemfile") + } + fn lock_path(&self) -> PathBuf { + self.root().join("Gemfile.lock") + } + fn installed_lib(&self) -> PathBuf { + self.root() + .join("vendor/bundle/ruby/3.4.0/gems/demo-gem-1.0.0/lib/demo_gem.rb") + } + fn copy_rel() -> String { + format!(".socket/vendor/gem/{GEM_UUID}/demo-gem-1.0.0") + } + fn vendored_lib(&self) -> PathBuf { + self.root().join(Self::copy_rel()).join("lib/demo_gem.rb") + } + fn state_path(&self) -> PathBuf { + self.root().join(".socket/vendor/state.json") + } +} + +fn gem_fixture() -> GemFixture { + let tmp = tempfile::tempdir().expect("tempdir"); + let root = tmp.path(); + std::fs::write(root.join("Gemfile"), GEM_GEMFILE).unwrap(); + std::fs::write(root.join("Gemfile.lock"), GEM_LOCK).unwrap(); + let home = root.join("vendor/bundle/ruby/3.4.0"); + std::fs::create_dir_all(home.join("gems/demo-gem-1.0.0/lib")).unwrap(); + std::fs::write(home.join("gems/demo-gem-1.0.0/lib/demo_gem.rb"), GEM_ORIG).unwrap(); + std::fs::create_dir_all(home.join("specifications")).unwrap(); + std::fs::write( + home.join("specifications/demo-gem-1.0.0.gemspec"), + GEM_GEMSPEC, + ) + .unwrap(); + GemFixture { tmp } +} + +/// Mount discovery (batch), per-package search, and the full view (inline +/// `blobContent`, so `scan --vendor` runs against the mock alone) for the +/// demo gem — the gem mirror of `scan_vendor_e2e::mount_patch_api`. +async fn mount_gem_patch_api(mock: &wiremock::MockServer) { + use base64::Engine as _; + use wiremock::matchers::{method, path, path_regex}; + use wiremock::{Mock, ResponseTemplate}; + + const ORG_SLUG: &str = "test-org"; + let before_hash = compute_git_sha256_from_bytes(GEM_ORIG); + let after_hash = compute_git_sha256_from_bytes(GEM_PATCHED); + let blob_b64 = base64::engine::general_purpose::STANDARD.encode(GEM_PATCHED); + Mock::given(method("POST")) + .and(path(format!("/v0/orgs/{ORG_SLUG}/patches/batch"))) + .respond_with(ResponseTemplate::new(200).set_body_json(json!({ + "packages": [{ + "purl": GEM_PURL, + "patches": [{ + "uuid": GEM_UUID, + "purl": GEM_PURL, + "tier": "free", + "cveIds": ["CVE-2026-0002"], + "ghsaIds": [], + "severity": "high", + "title": "gem vendor target" + }] + }], + "canAccessPaidPatches": false, + }))) + .mount(mock) + .await; + Mock::given(method("GET")) + .and(path_regex(format!( + "^/v0/orgs/{ORG_SLUG}/patches/by-package/.+$" + ))) + .respond_with(ResponseTemplate::new(200).set_body_json(json!({ + "patches": [{ + "uuid": GEM_UUID, + "purl": GEM_PURL, + "publishedAt": "2026-01-01T00:00:00Z", + "description": "gem vendor patch", + "license": "MIT", + "tier": "free", + "vulnerabilities": {} + }], + "canAccessPaidPatches": false, + }))) + .mount(mock) + .await; + Mock::given(method("GET")) + .and(path(format!("/v0/orgs/{ORG_SLUG}/patches/view/{GEM_UUID}"))) + .respond_with(ResponseTemplate::new(200).set_body_json(json!({ + "uuid": GEM_UUID, + "purl": GEM_PURL, + "publishedAt": "2026-01-01T00:00:00Z", + "files": { + "lib/demo_gem.rb": { + "beforeHash": before_hash, + "afterHash": after_hash, + "blobContent": blob_b64, + } + }, + "vulnerabilities": { + "GHSA-gem-vendor-test": { + "cves": ["CVE-2026-0002"], + "summary": "gem vendor vuln", + "severity": "high", + "description": "details" + } + }, + "description": "gem vendor patch", + "license": "MIT", + "tier": "free", + }))) + .mount(mock) + .await; +} + +fn run_scan_vendor(root: &Path, mock_uri: &str, extra: &[&str]) -> (i32, Value) { + let mut argv = vec![ + "scan", + "--json", + "--vendor", + "--yes", + "--api-url", + mock_uri, + "--api-token", + "fake-token", + "--org", + "test-org", + "--cwd", + root.to_str().unwrap(), + ]; + argv.extend_from_slice(extra); + let (code, stdout, stderr) = run_cli(root, &argv, &[]); + let env: Value = serde_json::from_str(stdout.trim()).unwrap_or_else(|e| { + panic!("scan --vendor --json must emit JSON: {e}\nstdout:\n{stdout}\nstderr:\n{stderr}") + }); + (code, env) +} + +/// `scan --vendor` end to end on a gem project: discover → download +/// (manifest written) → vendor lands the gem pair edit (Gemfile pin + +/// `path:`, lock PATH section + `(= …)!` DEPENDENCIES pin) and the patched +/// artifact dir — then reconcile auto-reverts once the manifest drops the +/// patch, byte-restoring both halves. +#[tokio::test] +async fn scan_vendor_gem_end_to_end_and_reconcile() { + let mock = wiremock::MockServer::start().await; + mount_gem_patch_api(&mock).await; + let fx = gem_fixture(); + + let (code, env) = run_scan_vendor(fx.root(), &mock.uri(), &[]); + assert_eq!(code, 0, "scan --vendor must succeed: {env:#}"); + assert_eq!(env["status"], "success", "envelope: {env:#}"); + assert_eq!(env["download"]["downloaded"], 1, "envelope: {env:#}"); + assert_eq!(env["vendor"]["summary"]["applied"], 1, "envelope: {env:#}"); + assert_eq!(env["vendor"]["summary"]["failed"], 0, "envelope: {env:#}"); + + // Manifest written by the download phase, keyed by the gem purl. + let manifest: Value = + serde_json::from_slice(&std::fs::read(fx.root().join(".socket/manifest.json")).unwrap()) + .unwrap(); + assert_eq!(manifest["patches"][GEM_PURL]["uuid"], GEM_UUID); + + // Artifact: patched bytes + the stub gemspec a path source needs. + assert_eq!( + std::fs::read(fx.vendored_lib()).unwrap(), + GEM_PATCHED, + "vendored lib must hold the patched bytes" + ); + assert_eq!( + std::fs::read_to_string( + fx.root() + .join(GemFixture::copy_rel()) + .join("demo-gem.gemspec") + ) + .unwrap(), + GEM_GEMSPEC, + "stub gemspec materialized from specifications/" + ); + + // The MANDATORY pair edit. + let gemfile = std::fs::read_to_string(fx.gemfile_path()).unwrap(); + assert!( + gemfile.contains(&format!( + "gem \"demo-gem\", \"1.0.0\", path: \"{}\"", + GemFixture::copy_rel() + )), + "Gemfile line not rewritten to the exact-pin + path: form:\n{gemfile}" + ); + let lock = std::fs::read_to_string(fx.lock_path()).unwrap(); + assert!( + lock.contains(&format!( + "PATH\n remote: {}\n specs:\n demo-gem (1.0.0)", + GemFixture::copy_rel() + )), + "canonical PATH section missing:\n{lock}" + ); + assert!( + lock.contains("\n demo-gem (= 1.0.0)!"), + "DEPENDENCIES pin missing:\n{lock}" + ); + + // The installed tree stays pristine (vendoring is not an in-place apply) + // and the ledger entry is manifest-tracked (not detached). + assert_eq!(std::fs::read(fx.installed_lib()).unwrap(), GEM_ORIG); + let state: Value = serde_json::from_slice(&std::fs::read(fx.state_path()).unwrap()).unwrap(); + assert_eq!(state["entries"][GEM_PURL]["ecosystem"], "gem"); + assert_eq!(state["entries"][GEM_PURL]["uuid"], GEM_UUID); + assert!( + state["entries"][GEM_PURL]["detached"].is_null(), + "manifest-mode entries are not detached: {state:#}" + ); + + // Idempotent re-run through the same JSON arm. + let gemfile_wired = std::fs::read(fx.gemfile_path()).unwrap(); + let lock_wired = std::fs::read(fx.lock_path()).unwrap(); + let (code, env2) = run_scan_vendor(fx.root(), &mock.uri(), &[]); + assert_eq!(code, 0, "re-run must succeed: {env2:#}"); + assert_eq!(env2["vendor"]["summary"]["applied"], 0, "{env2:#}"); + assert!( + env2["vendor"]["events"] + .as_array() + .unwrap() + .iter() + .any(|e| e["action"] == "skipped" && e["errorCode"] == "already_vendored"), + "re-run must be an already_vendored skip: {env2:#}" + ); + assert_eq!(std::fs::read(fx.gemfile_path()).unwrap(), gemfile_wired); + assert_eq!(std::fs::read(fx.lock_path()).unwrap(), lock_wired); + + // Reconcile: the patch dropped from the manifest is auto-reverted by the + // next plain vendor run — BOTH pair-edit halves byte-restored. + std::fs::write( + fx.root().join(".socket/manifest.json"), + b"{\"patches\": {}}\n", + ) + .unwrap(); + let (code, renv) = vendor_cli(fx.root(), &[]); + assert_eq!(code, 0, "reconcile-only run must exit 0: {renv:#}"); + let removed = find_event(&renv, "removed", Some("vendor_reconciled")); + assert_eq!(removed["purl"], GEM_PURL); + assert_eq!( + std::fs::read(fx.gemfile_path()).unwrap(), + GEM_GEMFILE.as_bytes(), + "reconcile must byte-restore the Gemfile" + ); + assert_eq!( + std::fs::read(fx.lock_path()).unwrap(), + GEM_LOCK.as_bytes(), + "reconcile must byte-restore Gemfile.lock" + ); + assert!( + !fx.root().join(".socket/vendor").exists(), + "the reconciled vendor tree must be fully pruned" + ); +} + +/// `scan --vendor --detached` on the gem project: no manifest is written, +/// the ledger entry is detached with the patch record embedded, the pair +/// edit still lands — and `vendor --revert` (the detached entry's only exit +/// path) byte-restores both files. +#[tokio::test] +async fn scan_vendor_gem_detached_writes_no_manifest_and_reverts() { + let mock = wiremock::MockServer::start().await; + mount_gem_patch_api(&mock).await; + let fx = gem_fixture(); + + let (code, env) = run_scan_vendor(fx.root(), &mock.uri(), &["--detached"]); + assert_eq!(code, 0, "scan --vendor --detached must succeed: {env:#}"); + assert_eq!(env["vendor"]["summary"]["applied"], 1, "envelope: {env:#}"); + + assert!( + !fx.root().join(".socket/manifest.json").exists(), + "detached mode must not write a manifest" + ); + assert!( + !fx.root().join(".socket/blobs").exists(), + "detached vendoring holds content in memory, never .socket/blobs" + ); + let state: Value = serde_json::from_slice(&std::fs::read(fx.state_path()).unwrap()).unwrap(); + assert_eq!(state["entries"][GEM_PURL]["detached"], json!(true)); + assert!( + state["entries"][GEM_PURL]["record"].is_object(), + "detached entries embed the patch record: {state:#}" + ); + assert_eq!(std::fs::read(fx.vendored_lib()).unwrap(), GEM_PATCHED); + let lock = std::fs::read_to_string(fx.lock_path()).unwrap(); + assert!( + lock.contains("\n demo-gem (= 1.0.0)!"), + "detached vendoring still lands the pair edit:\n{lock}" + ); + + // `--revert` is the detached entry's exit path: byte-restoration. + let (code, renv) = vendor_cli(fx.root(), &["--revert"]); + assert_eq!(code, 0, "revert must undo the detached entry: {renv:#}"); + assert_eq!( + std::fs::read(fx.gemfile_path()).unwrap(), + GEM_GEMFILE.as_bytes(), + "revert must byte-restore the Gemfile" + ); + assert_eq!( + std::fs::read(fx.lock_path()).unwrap(), + GEM_LOCK.as_bytes(), + "revert must byte-restore Gemfile.lock" + ); + assert!(!fx.root().join(".socket/vendor").exists()); +} diff --git a/crates/socket-patch-core/src/patch/redirect/mod.rs b/crates/socket-patch-core/src/patch/redirect/mod.rs index 3f337eed..c282b00f 100644 --- a/crates/socket-patch-core/src/patch/redirect/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/mod.rs @@ -1707,12 +1707,41 @@ fn rewrite_gem( if gem.is_empty() { return; } - let mut gemfile = files.get("Gemfile").cloned(); + // Bundler's modern manifest spelling: `gems.rb`/`gems.locked` wins over + // `Gemfile`/`Gemfile.lock` when both sit in one directory (bundler's + // `default_gemfile` tries gems.rb first — verified on bundler 4.0.15, + // which warns "Multiple gemfiles (gems.rb and Gemfile) detected ... + // bundler is ignoring them in favor of gems.rb and gems.locked"; same + // order as `setup::gem::discover_bundler_project`). DIVERGING spellings + // are ambiguous — the redirect would land in the file bundler reads while + // tooling pinned to the other keeps resolving upstream — so fail closed + // on the whole gem set. Identical spellings follow bundler: edit gems.rb. + let modern = files.contains_key("gems.rb"); + if modern && files.get("Gemfile").is_some_and(|c| files["gems.rb"] != *c) { + result.warnings.push(RewriteWarning { + code: "redirect_gem_gemfile_spellings_diverge".into(), + detail: "both gems.rb and Gemfile are present with different contents; bundler \ + reads gems.rb but the redirect cannot safely pick one — reconcile the \ + two spellings and re-run" + .into(), + }); + return; + } + let (gemfile_name, lock_name) = if modern { + ("gems.rb", "gems.locked") + } else { + ("Gemfile", "Gemfile.lock") + }; + let mut gemfile = files.get(gemfile_name).cloned(); let mut gemfile_changed = false; - let mut lock = files.get("Gemfile.lock").cloned(); + let mut lock = files.get(lock_name).cloned(); let mut lock_changed = false; // Static regex — compile once, not per-dependency (clippy: regex-in-loop). - let checksums_re = Regex::new(r"(?m)^CHECKSUMS$").unwrap(); + // `\r?` throughout the lock handling: a CRLF Gemfile.lock is legal to + // bundler (verified: `bundle check`/frozen install both accept one on + // 4.0.15), and without the tolerance the CHECKSUMS header never matched, + // misdiagnosing the lock as bundler <2.6. + let checksums_re = Regex::new(r"(?m)^CHECKSUMS(\r?)$").unwrap(); for dep in &gem { let Some(ov) = &dep.registry_override else { @@ -1757,7 +1786,7 @@ fn rewrite_gem( result.warnings.push(RewriteWarning { code: "redirect_gem_platform_unsupported".into(), detail: format!( - "Gemfile.lock CHECKSUMS carries platform-specific entries for {} {} — \ + "{lock_name} CHECKSUMS carries platform-specific entries for {} {} — \ the patch registry serves only the ruby platform gem; redirect skipped", dep.name, dep.version ), @@ -1805,7 +1834,7 @@ fn rewrite_gem( gf.replace_range(range, &ov.index_url); gemfile_changed = true; result.edits.push(FileEdit { - path: "Gemfile".into(), + path: gemfile_name.into(), kind: "redirect_gemfile_source_url".into(), action: "rewritten".into(), key: Some(dep.name.clone()), @@ -1897,7 +1926,7 @@ fn rewrite_gem( gf.replace_range(range, &block); gemfile_changed = true; result.edits.push(FileEdit { - path: "Gemfile".into(), + path: gemfile_name.into(), kind: "redirect_gemfile_source_block".into(), action: "rewritten".into(), key: Some(dep.name.clone()), @@ -1925,7 +1954,7 @@ fn rewrite_gem( *gf = format!("{gf}{sep}{block}\n"); gemfile_changed = true; result.edits.push(FileEdit { - path: "Gemfile".into(), + path: gemfile_name.into(), kind: "redirect_gemfile_source_block".into(), action: "added".into(), key: Some(dep.name.clone()), @@ -1946,7 +1975,8 @@ fn rewrite_gem( result.warnings.push(RewriteWarning { code: "redirect_gem_lock_without_source".into(), detail: format!( - "no Gemfile source redirect is in place for {} — CHECKSUMS pin skipped", + "no {gemfile_name} source redirect is in place for {} — CHECKSUMS pin \ + skipped", dep.name ), }); @@ -1957,13 +1987,16 @@ fn rewrite_gem( + ®ex::escape(&dep.name) + r" \(" + ®ex::escape(&dep.version) - + r"\)) sha256=([0-9a-f]+)$"), + + r"\)) sha256=([0-9a-f]+)(\r?)$"), ) .unwrap(); let new_val = format!("{} ({}) sha256={sha256}", dep.name, dep.version); // Already redirected (re-run): the CHECKSUMS line is at the // target value; recording an edit would grow the ledger forever. - if lk.contains(&format!("\n {new_val}\n")) || lk.ends_with(&format!("\n {new_val}")) { + let already_re = + Regex::new(&(String::from(r"(?m)^ ") + ®ex::escape(&new_val) + r"\r?$")) + .unwrap(); + if already_re.is_match(lk) { // no-op } else if let Some(m) = sum_line_re.captures(lk) { // The pre-edit line goes into the ledger as `original` so a @@ -1975,11 +2008,11 @@ fn rewrite_gem( m.get(2).unwrap().as_str() ); *lk = sum_line_re - .replace(lk, format!("${{1}} sha256={sha256}").as_str()) + .replace(lk, format!("${{1}} sha256={sha256}${{3}}").as_str()) .to_string(); lock_changed = true; result.edits.push(FileEdit { - path: "Gemfile.lock".into(), + path: lock_name.into(), kind: "redirect_gemfile_lock_checksum".into(), action: "rewritten".into(), key: Some(dep.name.clone()), @@ -1991,7 +2024,7 @@ fn rewrite_gem( .replace( lk, format!( - "CHECKSUMS\n {} ({}) sha256={sha256}", + "CHECKSUMS${{1}}\n {} ({}) sha256={sha256}${{1}}", dep.name, dep.version ) .as_str(), @@ -1999,7 +2032,7 @@ fn rewrite_gem( .to_string(); lock_changed = true; result.edits.push(FileEdit { - path: "Gemfile.lock".into(), + path: lock_name.into(), kind: "redirect_gemfile_lock_checksum".into(), action: "added".into(), key: Some(dep.name.clone()), @@ -2010,7 +2043,7 @@ fn rewrite_gem( result.warnings.push(RewriteWarning { code: "redirect_gem_no_checksums_section".into(), detail: format!( - "Gemfile.lock has no CHECKSUMS section (bundler <2.6) — cannot pin {}", + "{lock_name} has no CHECKSUMS section (bundler <2.6) — cannot pin {}", dep.name ), }); @@ -2025,22 +2058,23 @@ fn rewrite_gem( if gemfile_changed || lock_changed { result.warnings.push(RewriteWarning { code: "redirect_gem_frozen_install".into(), - detail: "Gemfile was repointed at the Socket patch registry but Gemfile.lock's \ - GEM section still records the upstream source; bundler rejects the pair \ - under frozen/deployment mode — run `bundle install` (unfrozen) once to \ - record the new source in Gemfile.lock" - .into(), + detail: format!( + "{gemfile_name} was repointed at the Socket patch registry but {lock_name}'s \ + GEM section still records the upstream source; bundler rejects the pair \ + under frozen/deployment mode — run `bundle install` (unfrozen) once to \ + record the new source in {lock_name}" + ), }); } if gemfile_changed { if let Some(gf) = gemfile { - result.files.insert("Gemfile".into(), gf); + result.files.insert(gemfile_name.into(), gf); } } if lock_changed { if let Some(lk) = lock { - result.files.insert("Gemfile.lock".into(), lk); + result.files.insert(lock_name.into(), lk); } } } @@ -4172,6 +4206,189 @@ mod tests { ); } + /// Bundler's modern `gems.rb`/`gems.locked` spelling must be redirected + /// exactly like the classic pair — before this, a gems.rb project was a + /// silent no-op (the rewriter keyed on the literal "Gemfile" names). + #[test] + fn gems_rb_pair_is_rewritten_with_modern_paths() { + let mut files = BTreeMap::new(); + files.insert( + "gems.rb".to_string(), + "source \"https://rubygems.org\"\n\ngem \"rails\", \"7.0.0\"\n".to_string(), + ); + files.insert( + "gems.locked".to_string(), + gem_lock(&format!(" rails (7.0.0) sha256={}", "2".repeat(64))), + ); + let r = rewrite_registry_redirect(&files, &[gem_override("rails", "7.0.0")]); + let gf = r.files.get("gems.rb").expect("gems.rb rewritten"); + assert!( + gf.contains( + "source \"https://patch.test/gem/tok/uuid/\" do\n gem \"rails\", \"7.0.0\"\nend" + ), + "source block lands in gems.rb: {gf}" + ); + let lk = r.files.get("gems.locked").expect("gems.locked rewritten"); + assert!( + lk.contains(&format!(" rails (7.0.0) sha256={}", "f".repeat(64))), + "CHECKSUMS pin lands in gems.locked: {lk}" + ); + assert!( + !r.files.contains_key("Gemfile") && !r.files.contains_key("Gemfile.lock"), + "classic spellings must not be invented: {:?}", + r.files.keys() + ); + // The ledger edits must name the files actually written, or a future + // revert restores the wrong pair. + assert!( + r.edits + .iter() + .any(|e| e.kind == "redirect_gemfile_source_block" && e.path == "gems.rb"), + "source-block edit keyed to gems.rb: {:?}", + r.edits + ); + assert!( + r.edits + .iter() + .any(|e| e.kind == "redirect_gemfile_lock_checksum" && e.path == "gems.locked"), + "lock edit keyed to gems.locked: {:?}", + r.edits + ); + } + + /// Both spellings present and byte-identical: follow bundler (which reads + /// gems.rb and ignores the Gemfile) — edit gems.rb, leave Gemfile alone. + #[test] + fn gems_rb_beats_identical_gemfile() { + let gemfile = "source \"https://rubygems.org\"\n\ngem \"rails\", \"7.0.0\"\n".to_string(); + let mut files = BTreeMap::new(); + files.insert("gems.rb".to_string(), gemfile.clone()); + files.insert("Gemfile".to_string(), gemfile); + let r = rewrite_registry_redirect(&files, &[gem_override("rails", "7.0.0")]); + assert!( + r.files.contains_key("gems.rb") && !r.files.contains_key("Gemfile"), + "bundler reads gems.rb, so only gems.rb may be edited: {:?}", + r.files.keys() + ); + } + + /// Both spellings present and DIVERGING: editing either is a guess (the + /// redirect could land in the file bundler ignores, or tooling pinned to + /// the classic name keeps resolving upstream). Fail closed with a warning. + #[test] + fn gems_rb_and_gemfile_diverging_fail_closed() { + let mut files = BTreeMap::new(); + files.insert( + "gems.rb".to_string(), + "source \"https://rubygems.org\"\n\ngem \"rails\", \"7.0.0\"\n".to_string(), + ); + files.insert( + "Gemfile".to_string(), + "source \"https://rubygems.org\"\n\ngem \"rails\", \"6.1.0\"\n".to_string(), + ); + let r = rewrite_registry_redirect(&files, &[gem_override("rails", "7.0.0")]); + assert!( + r.files.is_empty() && r.edits.is_empty(), + "diverging spellings must not be edited: files={:?} edits={:?}", + r.files.keys(), + r.edits + ); + assert!( + r.warnings + .iter() + .any(|w| w.code == "redirect_gem_gemfile_spellings_diverge"), + "fail-closed skip must warn: {:?}", + r.warnings + ); + } + + /// A CRLF Gemfile.lock is legal to bundler (`bundle check` and a frozen + /// install both accept one — verified on 4.0.15). The CHECKSUMS pin must + /// land in place, byte-preserving the `\r\n` endings — before this, the + /// `(?m)^…$` matchers never saw the `\r`-terminated lines and the lock + /// was misdiagnosed as bundler <2.6 (`redirect_gem_no_checksums_section`). + #[test] + fn gem_crlf_lock_checksum_pinned_preserving_crlf() { + let mut files = BTreeMap::new(); + files.insert( + "Gemfile".to_string(), + "source \"https://rubygems.org\"\n\ngem \"rails\", \"7.0.0\"\n".to_string(), + ); + files.insert( + "Gemfile.lock".to_string(), + gem_lock(&format!(" rails (7.0.0) sha256={}", "2".repeat(64))).replace('\n', "\r\n"), + ); + let r = rewrite_registry_redirect(&files, &[gem_override("rails", "7.0.0")]); + assert!( + !r.warnings + .iter() + .any(|w| w.code == "redirect_gem_no_checksums_section"), + "a CRLF CHECKSUMS section must be recognized: {:?}", + r.warnings + ); + let expected = + gem_lock(&format!(" rails (7.0.0) sha256={}", "f".repeat(64))).replace('\n', "\r\n"); + assert_eq!( + r.files.get("Gemfile.lock"), + Some(&expected), + "pin rewritten in place with every \\r\\n preserved" + ); + let edit = r + .edits + .iter() + .find(|e| e.kind == "redirect_gemfile_lock_checksum") + .expect("lock checksum edit recorded"); + assert_eq!( + edit.original, + Some(Value::String(format!( + "rails (7.0.0) sha256={}", + "2".repeat(64) + ))), + "recorded original carries no line-ending bytes" + ); + } + + /// CRLF lock whose CHECKSUMS section has no entry for the gem yet: the + /// added pin line must use the file's `\r\n` endings, not introduce a + /// lone `\n` into an otherwise-CRLF file. + #[test] + fn gem_crlf_lock_checksums_header_gains_crlf_entry() { + let mut files = BTreeMap::new(); + files.insert( + "Gemfile".to_string(), + "source \"https://rubygems.org\"\n\ngem \"rails\", \"7.0.0\"\n".to_string(), + ); + files.insert( + "Gemfile.lock".to_string(), + gem_lock(&format!(" nokogiri (1.16.0) sha256={}", "4".repeat(64))) + .replace('\n', "\r\n"), + ); + let r = rewrite_registry_redirect(&files, &[gem_override("rails", "7.0.0")]); + let lk = r.files.get("Gemfile.lock").expect("lock rewritten"); + assert!( + lk.contains(&format!( + "CHECKSUMS\r\n rails (7.0.0) sha256={}\r\n", + "f".repeat(64) + )), + "added pin keeps the CRLF endings: {lk:?}" + ); + + // Re-run on the rewritten pair: recognizing the at-target CRLF line + // must be a no-op (the ledger would otherwise grow forever). + files.insert("Gemfile.lock".to_string(), lk.clone()); + files.insert( + "Gemfile".to_string(), + r.files.get("Gemfile").expect("Gemfile rewritten").clone(), + ); + let second = rewrite_registry_redirect(&files, &[gem_override("rails", "7.0.0")]); + assert!( + second.files.is_empty() && second.edits.is_empty(), + "CRLF re-run must be a no-op: files={:?} edits={:?}", + second.files.keys(), + second.edits + ); + } + /// An unparseable package-lock.json must surface a warning, not silently /// skip the npm redirect entirely (missing-lockfile already warns; a /// corrupt lockfile is strictly worse and was silent). From 707bbf8d1e53f960c82a5c4595e3ed4044c2f96d Mon Sep 17 00:00:00 2001 From: Mikola Lysenko Date: Thu, 13 Aug 2026 18:14:55 -0700 Subject: [PATCH 4/4] fix(redirect): judge gems.rb/Gemfile divergence on the redirect footprint residue, not raw bytes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Run 1 on byte-identical gems.rb/Gemfile twins edits only gems.rb (the file bundler reads), so the raw-byte divergence guard trapped every later run — the rotated-grant URL refresh included — behind redirect_gem_gemfile_spellings_diverge, a divergence the rewriter itself created. Compare redirect-footprint residues instead: erase the managed Socket source block (rotating grant segments wildcarded) and the dep's own gem declaration line from both spellings before judging. Rebased onto d063cb1 (CRLF-tolerant block recognizer): the residue eraser now accepts `do\r?\n … end\r?` too, mirroring the recognizer — a core.autocrlf checkout rewrites run 1's LF block to CRLF, and a block the recognizer accepts must also be erased here or the re-run is trapped behind the divergence warning before it can reach the recognizer. Pinned by gems_rb_crlf_twins_rerun_is_no_op_and_rotated_grant_refreshes (verified red with an LF-only residue eraser). Co-Authored-By: Claude Fable 5 --- .../src/patch/redirect/mod.rs | 335 +++++++++++++++++- docs/ecosystems.md | 2 +- 2 files changed, 322 insertions(+), 15 deletions(-) diff --git a/crates/socket-patch-core/src/patch/redirect/mod.rs b/crates/socket-patch-core/src/patch/redirect/mod.rs index c282b00f..ecee7c3e 100644 --- a/crates/socket-patch-core/src/patch/redirect/mod.rs +++ b/crates/socket-patch-core/src/patch/redirect/mod.rs @@ -1698,6 +1698,61 @@ fn gem_tail_source_option(tail: &str) -> Option<&'static str> { .find(|tok| code.contains(tok)) } +/// A dep's Socket index URL as a regex source with the per-request rotating +/// segments (grant token, patch uuid) wildcarded — an exact-URL pattern +/// misses the URL a previous run wrote under an older grant. +fn gem_index_url_pattern(dep: &DepOverride, index_url: &str) -> String { + let mut url_pat = regex::escape(index_url); + for rotating in [&dep.token, &dep.patch_uuid] { + if !rotating.is_empty() { + url_pat = url_pat.replace(®ex::escape(&format!("/{rotating}/")), "/[^/\"]+/"); + } + } + url_pat +} + +/// A gemfile spelling with the redirect's own footprint erased: every managed +/// Socket `source "…" do … end` block for a redirected dep (rotating grant +/// segments wildcarded) and the dep's own `gem` declaration line. The +/// gems.rb/Gemfile divergence guard compares these residues rather than raw +/// bytes: run 1 on byte-identical twins edits only gems.rb (the file bundler +/// reads), so a raw comparison would trap every later run — the rotated-grant +/// URL refresh included — behind `redirect_gem_gemfile_spellings_diverge`, a +/// divergence the rewriter itself created. Trailing whitespace is trimmed (a +/// block appended to a newline-less file adds a final newline the other +/// spelling never had). `\r?` mirrors the block recognizer in `rewrite_gem`: +/// a `core.autocrlf` checkout rewrites run 1's LF block to CRLF, and a block +/// the recognizer accepts must also be erased here or the re-run is trapped +/// behind the divergence warning before it can reach the recognizer. +fn gem_spelling_residue(content: &str, deps: &[&DepOverride]) -> String { + let mut residue = content.to_string(); + for dep in deps { + let Some(ov) = &dep.registry_override else { + continue; + }; + if ov.kind != "rubygems-compact-index" { + continue; + } + let block_re = Regex::new( + &(String::from(r#"(?m)^source ""#) + + &gem_index_url_pattern(dep, &ov.index_url) + + r#"" do\r?\n gem ["']"# + + ®ex::escape(&dep.name) + + r#"["'][^\n]*\nend\r?\n?"#), + ) + .unwrap(); + residue = block_re.replace_all(&residue, "").into_owned(); + let decl_re = Regex::new( + &(String::from(r#"(?m)^[ \t]*gem\b[^\n]*["']"#) + + ®ex::escape(&dep.name) + + r#"["'][^\n]*\n?"#), + ) + .unwrap(); + residue = decl_re.replace_all(&residue, "").into_owned(); + } + residue.trim_end().to_string() +} + fn rewrite_gem( files: &BTreeMap, overrides: &[DepOverride], @@ -1715,9 +1770,17 @@ fn rewrite_gem( // order as `setup::gem::discover_bundler_project`). DIVERGING spellings // are ambiguous — the redirect would land in the file bundler reads while // tooling pinned to the other keeps resolving upstream — so fail closed - // on the whole gem set. Identical spellings follow bundler: edit gems.rb. + // on the whole gem set. Divergence is judged on the redirect-footprint + // residue (`gem_spelling_residue`), NOT raw bytes: run 1 on identical + // twins edits only gems.rb (following bundler), so a raw comparison would + // trap every later run behind the divergence the rewriter itself created. + // Identical spellings follow bundler: edit gems.rb. let modern = files.contains_key("gems.rb"); - if modern && files.get("Gemfile").is_some_and(|c| files["gems.rb"] != *c) { + if modern + && files.get("Gemfile").is_some_and(|c| { + gem_spelling_residue(&files["gems.rb"], &gem) != gem_spelling_residue(c, &gem) + }) + { result.warnings.push(RewriteWarning { code: "redirect_gem_gemfile_spellings_diverge".into(), detail: "both gems.rb and Gemfile are present with different contents; bundler \ @@ -1805,13 +1868,7 @@ fn rewrite_gem( // run would wrap the gem line inside it — nesting source blocks. // Wildcard the rotating segments instead (mirrors the CHECKSUMS // at-target guard below). - let mut url_pat = regex::escape(&ov.index_url); - for rotating in [&dep.token, &dep.patch_uuid] { - if !rotating.is_empty() { - url_pat = - url_pat.replace(®ex::escape(&format!("/{rotating}/")), "/[^/\"]+/"); - } - } + let url_pat = gem_index_url_pattern(dep, &ov.index_url); // `\r?\n`: the rewriter emits LF, but a `core.autocrlf` checkout // rewrites the working tree to CRLF — the guard must still // recognize the block there, or the indented `gem` line inside @@ -4272,19 +4329,22 @@ mod tests { ); } - /// Both spellings present and DIVERGING: editing either is a guess (the - /// redirect could land in the file bundler ignores, or tooling pinned to - /// the classic name keeps resolving upstream). Fail closed with a warning. + /// Both spellings present and DIVERGING outside the redirect's own + /// footprint (an unrelated gem only one file declares): editing either is + /// a guess (the redirect could land in the file bundler ignores, or + /// tooling pinned to the classic name keeps resolving upstream). Fail + /// closed with a warning. #[test] fn gems_rb_and_gemfile_diverging_fail_closed() { let mut files = BTreeMap::new(); files.insert( "gems.rb".to_string(), - "source \"https://rubygems.org\"\n\ngem \"rails\", \"7.0.0\"\n".to_string(), + "source \"https://rubygems.org\"\n\ngem \"rails\", \"7.0.0\"\ngem \"puma\", \"6.0.0\"\n" + .to_string(), ); files.insert( "Gemfile".to_string(), - "source \"https://rubygems.org\"\n\ngem \"rails\", \"6.1.0\"\n".to_string(), + "source \"https://rubygems.org\"\n\ngem \"rails\", \"7.0.0\"\n".to_string(), ); let r = rewrite_registry_redirect(&files, &[gem_override("rails", "7.0.0")]); assert!( @@ -4302,6 +4362,253 @@ mod tests { ); } + /// Divergence confined to the redirected dep's OWN declaration line is + /// tolerated: the rewriter canonicalizes that line into the managed block + /// either way, and bundler reads gems.rb regardless (verified on 4.0.15, + /// which warns it is ignoring the Gemfile). Only divergence outside the + /// redirect's footprint is ambiguous enough to fail closed on. + #[test] + fn gems_rb_divergence_only_in_redirected_dep_line_proceeds() { + let mut files = BTreeMap::new(); + files.insert( + "gems.rb".to_string(), + "source \"https://rubygems.org\"\n\ngem \"rails\", \"7.0.0\"\n".to_string(), + ); + files.insert( + "Gemfile".to_string(), + "source \"https://rubygems.org\"\n\ngem \"rails\", \"6.1.0\"\n".to_string(), + ); + let r = rewrite_registry_redirect(&files, &[gem_override("rails", "7.0.0")]); + assert!( + !r.warnings + .iter() + .any(|w| w.code == "redirect_gem_gemfile_spellings_diverge"), + "the redirected dep's own line is not ambient divergence: {:?}", + r.warnings + ); + assert!( + r.files.contains_key("gems.rb") && !r.files.contains_key("Gemfile"), + "redirect proceeds on the file bundler reads: {:?}", + r.files.keys() + ); + } + + /// Run 1 on byte-identical twins edits only gems.rb (bundler's file), + /// which makes the pair diverge on raw bytes. The divergence guard judges + /// the redirect-footprint residue instead: feeding run 1's output back + /// must be a plain no-op re-run, not a + /// `redirect_gem_gemfile_spellings_diverge` trap that blocks every later + /// run against the state run 1 itself created. + #[test] + fn gems_rb_identical_twins_rerun_is_a_no_op_not_a_diverge_trap() { + let gemfile = "source \"https://rubygems.org\"\n\ngem \"rails\", \"7.0.0\"\n".to_string(); + let mut files = BTreeMap::new(); + files.insert("gems.rb".to_string(), gemfile.clone()); + files.insert("Gemfile".to_string(), gemfile); + files.insert( + "gems.locked".to_string(), + gem_lock(&format!(" rails (7.0.0) sha256={}", "2".repeat(64))), + ); + let ovr = gem_override("rails", "7.0.0"); + let first = rewrite_registry_redirect(&files, std::slice::from_ref(&ovr)); + assert!( + first.files.contains_key("gems.rb") && first.files.contains_key("gems.locked"), + "run 1 lands on the modern pair: files={:?} warnings={:?}", + first.files.keys(), + first.warnings + ); + for (name, content) in first.files { + files.insert(name, content); + } + let second = rewrite_registry_redirect(&files, std::slice::from_ref(&ovr)); + assert!( + !second + .warnings + .iter() + .any(|w| w.code == "redirect_gem_gemfile_spellings_diverge"), + "the divergence run 1 itself created must not trap run 2: {:?}", + second.warnings + ); + assert!( + second.files.is_empty() && second.edits.is_empty(), + "same-grant re-run is a no-op: files={:?} edits={:?}", + second.files.keys(), + second.edits + ); + } + + /// The identical-twins re-run with a ROTATED grant (the token/uuid URL + /// segments rotate per request) must still reach the in-place URL + /// refresh — with a raw-byte divergence guard, run 1's edit tripped the + /// trap and the redirect went permanently stale under the old grant. + #[test] + fn gems_rb_identical_twins_rerun_refreshes_rotated_grant_url() { + fn ov(token: &str) -> DepOverride { + let mut o = gem_override("rails", "7.0.0"); + o.token = token.into(); + if let Some(r) = o.registry_override.as_mut() { + r.index_url = format!("https://patch.test/gem/{token}/uuid/"); + } + o + } + let gemfile = "source \"https://rubygems.org\"\n\ngem \"rails\", \"7.0.0\"\n".to_string(); + let mut files = BTreeMap::new(); + files.insert("gems.rb".to_string(), gemfile.clone()); + files.insert("Gemfile".to_string(), gemfile); + let first = rewrite_registry_redirect(&files, &[ov("tok-one")]); + for (name, content) in first.files { + files.insert(name, content); + } + let second = rewrite_registry_redirect(&files, &[ov("tok-two")]); + assert!( + !second + .warnings + .iter() + .any(|w| w.code == "redirect_gem_gemfile_spellings_diverge"), + "run 1's own edit must not read as divergence: {:?}", + second.warnings + ); + let out = second + .files + .get("gems.rb") + .expect("rotated grant refreshes gems.rb"); + assert!( + out.contains( + "source \"https://patch.test/gem/tok-two/uuid/\" do\n gem \"rails\", \"7.0.0\"\nend" + ), + "URL refreshed in place: {out}" + ); + assert!(!out.contains("tok-one"), "old grant token gone: {out}"); + assert!( + second + .edits + .iter() + .any(|e| e.kind == "redirect_gemfile_source_url" && e.path == "gems.rb"), + "refresh recorded against gems.rb: {:?}", + second.edits + ); + } + + /// Twins where the redirected dep is TRANSITIVE (undeclared): run 1 + /// appends a source block to gems.rb — a footprint shape the residue + /// comparison must also erase, including the final newline the append + /// adds to a newline-less file. + #[test] + fn gems_rb_identical_twins_rerun_after_appended_block_is_no_op() { + let gemfile = "source \"https://rubygems.org\"\n\ngem \"rack\", \"3.0.0\"".to_string(); + let mut files = BTreeMap::new(); + files.insert("gems.rb".to_string(), gemfile.clone()); + files.insert("Gemfile".to_string(), gemfile); + let ovr = gem_override("rails", "7.0.0"); + let first = rewrite_registry_redirect(&files, std::slice::from_ref(&ovr)); + assert!( + first + .files + .get("gems.rb") + .is_some_and(|gf| gf.contains("source \"https://patch.test/gem/tok/uuid/\" do")), + "run 1 appends the block for the undeclared dep: {:?}", + first.files + ); + for (name, content) in first.files { + files.insert(name, content); + } + let second = rewrite_registry_redirect(&files, std::slice::from_ref(&ovr)); + assert!( + !second + .warnings + .iter() + .any(|w| w.code == "redirect_gem_gemfile_spellings_diverge"), + "an appended block is the redirect's own footprint, not divergence: {:?}", + second.warnings + ); + assert!( + second.files.is_empty() && second.edits.is_empty(), + "re-run is a no-op: files={:?} edits={:?}", + second.files.keys(), + second.edits + ); + } + + /// The block recognizer accepts a CRLF Socket source block (a + /// `core.autocrlf` checkout rewrites run 1's LF output), so the residue + /// comparison must erase that CRLF spelling too: after the checkout + /// rewrites BOTH twins to CRLF, only gems.rb carries the block — if the + /// residue regex stays LF-only the block survives into gems.rb's residue + /// and every later run (the rotated-grant URL refresh included) is + /// trapped behind `redirect_gem_gemfile_spellings_diverge`. + #[test] + fn gems_rb_crlf_twins_rerun_is_no_op_and_rotated_grant_refreshes() { + fn ov(token: &str) -> DepOverride { + let mut o = gem_override("rails", "7.0.0"); + o.token = token.into(); + if let Some(r) = o.registry_override.as_mut() { + r.index_url = format!("https://patch.test/gem/{token}/uuid/"); + } + o + } + // gems.rb exactly as run 1 wrote it, after a CRLF checkout; the + // Gemfile twin got the same CRLF treatment but never had the block. + let mut files = BTreeMap::new(); + files.insert( + "gems.rb".to_string(), + "source \"https://rubygems.org\"\r\n\r\n\ + source \"https://patch.test/gem/tok-one/uuid/\" do\r\n \ + gem \"rails\", \"7.0.0\"\r\nend\r\n" + .to_string(), + ); + files.insert( + "Gemfile".to_string(), + "source \"https://rubygems.org\"\r\n\r\ngem \"rails\", \"7.0.0\"\r\n".to_string(), + ); + + // Same grant: recognized in place, a true no-op — not a diverge trap. + let same = rewrite_registry_redirect(&files, &[ov("tok-one")]); + assert!( + !same + .warnings + .iter() + .any(|w| w.code == "redirect_gem_gemfile_spellings_diverge"), + "the CRLF block is the redirect's own footprint, not divergence: {:?}", + same.warnings + ); + assert!( + same.files.is_empty() && same.edits.is_empty(), + "same-grant re-run on CRLF twins is a no-op: files={:?} edits={:?}", + same.files.keys(), + same.edits + ); + + // Rotated grant: URL refreshed in place inside gems.rb, never nested. + let rotated = rewrite_registry_redirect(&files, &[ov("tok-two")]); + assert!( + !rotated + .warnings + .iter() + .any(|w| w.code == "redirect_gem_gemfile_spellings_diverge"), + "rotated grant must reach the refresh, not the diverge trap: {:?}", + rotated.warnings + ); + let out = rotated + .files + .get("gems.rb") + .expect("rotated grant refreshes gems.rb on a CRLF checkout"); + assert_eq!( + out.matches("source \"https://patch.test/gem/").count(), + 1, + "exactly one Socket source block, never nested: {out}" + ); + assert!(!out.contains("tok-one"), "old grant token gone: {out}"); + assert!( + out.contains("source \"https://patch.test/gem/tok-two/uuid/\" do\r\n"), + "existing CRLF block body left intact: {out}" + ); + assert!( + !rotated.files.contains_key("Gemfile"), + "bundler reads gems.rb; the Gemfile twin stays untouched: {:?}", + rotated.files.keys() + ); + } + /// A CRLF Gemfile.lock is legal to bundler (`bundle check` and a frozen /// install both accept one — verified on 4.0.15). The CHECKSUMS pin must /// land in place, byte-preserving the `\r\n` endings — before this, the diff --git a/docs/ecosystems.md b/docs/ecosystems.md index 1313c4eb..adf939c5 100644 --- a/docs/ecosystems.md +++ b/docs/ecosystems.md @@ -17,7 +17,7 @@ The backticked slug in each row is the value `-e`/`--ecosystems` accepts (e.g. | npm (`npm`) — pnpm / yarn / berry / bun | ✅ any install layout; `setup` postinstall hook | ✅ five lockfile flavors: package-lock, yarn classic, yarn berry (node-modules linker; PnP refused), pnpm v9, bun `bun.lock` (binary `bun.lockb` refused with a `--save-text-lockfile` pointer). Rush monorepos refused (`vendor_rush_unsupported`) — see [Rush notes](#npm-rush-monorepos) | ✅ package-lock / npm-shrinkwrap, pnpm-lock.yaml, yarn classic, yarn berry, bun — berry and bun carry constraints, see [npm hosted-mode notes](#npm-hosted-mode-notes) | | PyPI (`pypi`) — uv / poetry / pdm / pipenv / pip | ✅ `.pth` startup hook via `setup` | ✅ five lockfile flavors: uv, poetry, pdm, pipenv (lock rewired, but pipenv doesn't hash-check file entries — `vendor_integrity_unverified` warning; the committed wheel bytes are the protection), and requirements.txt (consumed by pip or `uv pip`) | ✅ requirements.txt + uv.lock. **poetry / pdm / pipenv locks are not rewritten** — use vendored | | Cargo (`cargo`) | ✅ in-place + `.cargo-checksum.json` rewrite (shared registry-cache caveat — see [Cargo: shared registry cache](#cargo-shared-registry-cache)) | ✅ `[patch.crates-io]` path entry | ✅ per-patch sparse registry (`[registries.socket-patch-]` + Cargo.lock source/checksum) | -| RubyGems (`gem`) | ✅ Bundler plugin via `setup` | ✅ Gemfile + Gemfile.lock path pair | ✅ per-dep `source` block; the `CHECKSUMS` pin needs bundler ≥ 2.6 (older locks get a `redirect_gem_no_checksums_section` warning) | +| RubyGems (`gem`) | ✅ Bundler plugin via `setup` | ✅ Gemfile + Gemfile.lock path pair (`Gemfile` spelling only — a `gems.rb` project cannot vendor yet) | ✅ per-dep `source` block — edits `gems.rb` + `gems.locked` when present (bundler prefers them over `Gemfile`; spellings that diverge beyond Socket's own edits fail closed with `redirect_gem_gemfile_spellings_diverge`); the `CHECKSUMS` pin needs bundler ≥ 2.6 (older locks get a `redirect_gem_no_checksums_section` warning) | | Go (`golang`) | ✅ `go.mod` `replace` → `.socket/go-patches/` — see [Go: directory replaces and go.sum](#go-directory-replaces-and-gosum) | ✅ `replace` → the committed vendor tree | ❌ **not possible** — sumdb, module-path identity, and default-GOPROXY leakage each rule it out; see [golang-hosted-no-go.md](design/golang-hosted-no-go.md). **Use vendored** (`redirect_golang_unsupported` names the remedy) | | Maven (`maven`) | ✅ apply-only (no `setup` hook — reports `no_files`); in-place jar patching leaves the `~/.m2` checksum sidecars stale — prefer vendored / hosted, see [Maven & NuGet caveats](#maven--nuget-caveats) | ✅ committed maven2 `file://` repository. A root pom declaring `` (multi-module aggregator) is refused (`vendor_maven_multimodule_unsupported`), and a gradle-only project is refused (`vendor_gradle_unsupported`) | ✅ **pom projects only, fail-closed** — the patched jar is pinned at a Socket-only `-socket.` suffix; `${property}` versions are refused; Gradle gets a manual `exclusiveContent` snippet — see [Maven & NuGet caveats](#maven--nuget-caveats) | | NuGet (`nuget`) | ✅ apply-only (no `setup` hook — reports `no_files`); in-place patching deletes `.nupkg.metadata` and advises on the `.nupkg.sha512` tamper-evidence sidecar — prefer vendored / hosted, see [Maven & NuGet caveats](#maven--nuget-caveats) | ✅ committed folder feed + `packageSourceMapping` + `packages.lock.json` contentHash pin | ✅ `nuget.config` source + source-mapping, `packages.lock.json` contentHash rewrite. See the locked-mode note in [Maven & NuGet caveats](#maven--nuget-caveats) |