Skip to content

test(hosted): retire the gem catalog pin — production unpublished its last free gem patch (activestorage@7.0.2.2) - #199

Open
Mikola Lysenko (mikolalysenko) wants to merge 1 commit into
mainfrom
test/hosted-gem-catalog-replacement
Open

test(hosted): retire the gem catalog pin — production unpublished its last free gem patch (activestorage@7.0.2.2)#199
Mikola Lysenko (mikolalysenko) wants to merge 1 commit into
mainfrom
test/hosted-gem-catalog-replacement

Conversation

@mikolalysenko

@mikolalysenko Mikola Lysenko (mikolalysenko) commented Aug 15, 2026

Copy link
Copy Markdown
Collaborator

What production unpublished

The free-tier gem patch this suite pinned — pkg:gem/activestorage@7.0.2.2,
uuid 2535d43d-67ce-4944-be27-c19e113997fb (GHSA-w749-p3v6-hccq) — is gone
from production: the versioned and version-less /patch/by-package probes both
return {"patches": []} and /patch/view/2535d43d-… now 404s (while the
npm pin's /patch/view still 200s, so it is a real unpublish, not an API
change). That broke three tests on every branch:

  • preflight_required_patches_are_published
  • canary_patches_name_advisories_so_merge_state_is_inferable
  • gem_bundler_hosted_redirect_and_known_install_defect (0 redirects)

Why nothing replaced it: there is no free gem patch left to pin

The preflight's instruction is "pick a replacement", but a replacement has to
exist. It does not:

  • probed every gem name in ruby-advisory-db (464 packages) version-lessly
    against patches-api.socket.dev/patch/by-package — zero free patches;
  • sanity-checked the probe against the suite's other pins the same way:
    pkg:npm/minimist → 7 patches, pkg:pypi/urllib3 → 71, pkg:cargo/traitobject → 4,
    pkg:npm/lodash → 48 (all tier: "free"), so the route works and the gem
    emptiness is real;
  • pkg:gem/activestorage@7.0.2.2?platform=ruby (the qualified form) also
    returns nothing.

activestorage@7.0.2.2 was the only free gem patch production ever
published (see #141's catalog); withdrawing it emptied the ecosystem.

What this PR does instead: gem joins the unpublished-ecosystems canary

The suite already has an honest pattern for exactly this state — maven, nuget
and composer implement hosted mode but have no free patches, and
canary_unpublished_ecosystems probes them every run so coverage can be added
the moment one lights up. Gem now follows it:

  • catalog: GEM_* constants removed; preflight and the merge-state canary
    now cover npm / PyPI / cargo;
  • canary: UNPUBLISHED_ECOSYSTEMS gains a gem entry whose first
    candidate is the withdrawn pin itself (pkg:gem/activestorage, plus
    rails-html-sanitizer / nokogiri / rack), so the canary reports the most
    likely republish fastest;
  • gem leg: gem_bundler_hosted_redirect_and_known_install_defect and its
    helpers (gem_lock_checksum, installed_gem_dir, gem_registry_base,
    http_probe, published_patch_files) are retired — with no published patch
    neither its known-defect arm nor its success arm can execute (scan --mode hosted redirects 0 and every assertion is unreachable). They live in this
    PR's deletion diff for restoration; the docs say exactly what to resurrect.
    The gem hosted rewrite grammar stays covered by e2e_redirect_gem_build.rs
    against a mock;
  • ci: the hosted-e2e job's ruby/setup-ruby step is removed (nothing in
    the suite needs ruby now), with a comment pointing at the restoration path;
  • docs: docs/testing/hosted-production-e2e.md — catalog table, coverage
    table, a "retired bundler leg" section with the restoration recipe, the
    known-defect entry kept for the record, SOCKET_PATCH_HOSTED_E2E_GEM_STRICT
    knob removed with the leg.

The known compact-index server defect (/info/<gem> missing runtime deps,
bundler APIResponseMismatchError) is still documented; it is believed to
still stand server-side, but nothing free-tier remains to probe it against.

Which arm does the gem leg take now?

Neither — that is the point. Without a published patch the leg cannot reach
either its known-defect arm or its content-verified success arm; coverage is
the canary until production republishes, after which the leg (and its
probe-the-server tolerance) should be restored from this PR's deletion.

activestorage@5.2.0 (e2e_gem.rs) is ALSO unpublished — follow-up needed

pkg:gem/activestorage@5.2.0, uuid 4bf7fe0b-dc57-4ea8-945f-bc4a04c47a15,
used by the e2e_gem.rs lifecycle tests, is gone too (/patch/view
404s, by-package empty). Every gem probe returned empty, so there is no
replacement pin for that suite either. Not fixed here (out of scope, and
e2e_gem is deliberately not in the PR matrix — CI unaffected); it needs the
same treatment as a follow-up. The vendored twin e2e_vendored_production.rs
pins the same withdrawn 7.0.2.2 patch
(also not in CI) and needs the same
follow-up.

Live verification

cargo test -p socket-patch-cli --features docker-e2e,setup-e2e --test e2e_hosted_production -- --ignored --nocapture against real production, all
toolchains present (macOS; npm/pnpm/yarn/corepack/bun/uv/cargo/go):

test preflight_required_patches_are_published ... ok
test canary_patches_name_advisories_so_merge_state_is_inferable ... ok
test canary_published_at_is_a_patch_date_not_a_package_date ... ok
test npm_package_lock_hosted_install_proof ... ok
test npm_shrinkwrap_hosted_redirect ... ok
test pnpm_hosted_install_proof ... ok        (known pnpm-11 --trust-lockfile gap reported, as before)
test yarn_classic_hosted_install_proof ... ok
test yarn_berry_hosted_install_proof ... ok
test bun_hosted_install_proof ... ok
test pypi_requirements_txt_hosted_install_proof ... ok
test pypi_uv_lock_hosted_install_proof ... ok
test cargo_hosted_install_proof ... ok
test golang_hosted_is_refused_by_design ... ok
test deno_hosted_is_unsupported ... ok
test canary_unpublished_ecosystems ... ok
  ("gem / maven / nuget / composer still have no free-tier published patches")

test result: ok. 15 passed; 0 failed; 0 ignored; 0 measured; 5 filtered out
(the 5 filtered are cache_env.rs's non-ignored unit tests)

The three previously-broken tests pass; every install-proof leg stays green.
(First attempt of the run hit transport flakes on patches-api.socket.dev
failures said "error sending request", not unpublish — and was rerun per the
suite's own 503/flake policy.)

🤖 Generated with Claude Code


Note

Low Risk
Test and CI documentation changes only; no production CLI or runtime behavior is modified.

Overview
Production withdrew the suite’s only free-tier RubyGems pin (activestorage@7.0.2.2), leaving zero free gem patches to test against. This PR retires the live bundler hosted e2e leg instead of leaving preflight and install tests red or vacuous.

e2e_hosted_production.rs drops GEM_* catalog constants, gem-only helpers (gem_lock_checksum, installed_gem_dir, gem_registry_base, http_probe, published_patch_files), and the large gem_bundler_hosted_redirect_and_known_install_defect test. Preflight and advisory canaries now cover npm, PyPI, and Cargo only. Gem is added to UNPUBLISHED_ECOSYSTEMS (with activestorage first among probe candidates) so canary_unpublished_ecosystems can signal when a free patch returns.

CI removes ruby/setup-ruby from the hosted-e2e job, with a comment to restore bundler ≥ 2.6 when the leg comes back.

docs/testing/hosted-production-e2e.md updates the patch catalog and coverage tables, documents the retired bundler leg and restoration steps from git history, removes SOCKET_PATCH_HOSTED_E2E_GEM_STRICT, and notes the known compact-index server defect is unprobed until a free gem patch exists again. Mock coverage remains in e2e_redirect_gem_build.rs.

Reviewed by Cursor Bugbot for commit a382398. Configure here.

… last free gem patch

Production withdrew pkg:gem/activestorage@7.0.2.2 (uuid
2535d43d-67ce-4944-be27-c19e113997fb, GHSA-w749-p3v6-hccq) — the only
free-tier gem patch it ever published. /patch/view/<uuid> now 404s (the npm
pin's still 200s) and a version-less probe of every gem name in
ruby-advisory-db (464 packages) returns zero free patches, so there is no
honest replacement pin: the preflight's "pick a replacement" instruction has
nothing to pick.

Gem therefore joins the suite's existing pattern for patch-less ecosystems
(maven / nuget / composer):

* drop the GEM_* catalog entries from the preflight and the merge-state
  canary (npm / PyPI / cargo remain);
* add gem to UNPUBLISHED_ECOSYSTEMS — first candidate is the withdrawn pin
  itself — so canary_unpublished_ecosystems reports the moment production
  republishes a gem patch;
* retire gem_bundler_hosted_redirect_and_known_install_defect and its
  gem-only helpers (gem_lock_checksum, installed_gem_dir, gem_registry_base,
  http_probe, published_patch_files): with no published patch neither its
  known-defect arm nor its success arm can execute. Restoration recipe is in
  docs/testing/hosted-production-e2e.md; the rewrite grammar stays covered by
  e2e_redirect_gem_build.rs against a mock;
* ci: drop the hosted-e2e job's now-unneeded ruby/setup-ruby step;
* docs: catalog + coverage tables updated, retired-leg section added, the
  compact-index server-defect entry kept for the record, and the
  SOCKET_PATCH_HOSTED_E2E_GEM_STRICT knob removed with the leg.

Verified live against production: 15 passed / 0 failed (the three
previously-broken tests pass; every install-proof leg stays green).

NOTE: e2e_gem.rs's pkg:gem/activestorage@5.2.0 pin
(4bf7fe0b-dc57-4ea8-945f-bc4a04c47a15) and e2e_vendored_production.rs's pin
of the same 7.0.2.2 patch are ALSO unpublished — separate follow-up; neither
suite is in CI.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant