diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index 100838d2..75fef147 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -397,6 +397,7 @@ per service outcome: | integrity mismatch | cargo/maven/nuget: **refuse** (`vendor_prebuilt_integrity_mismatch`) — tampered bytes never fall back; other ecosystems (to be aligned): local build + `vendor_prebuilt_integrity_mismatch` | refuse (cargo/maven/nuget: `vendor_prebuilt_integrity_mismatch`; others: `vendor_prebuilt_required`) | | still building (`pending_build` / serve 408) | local build + `vendor_prebuilt_pending` | refuse | | not built / withdrawn / not found / no usable artifact | local build (quiet) | refuse | +| gem stub gemspec missing / invalid | local build + `vendor_prebuilt_stub_missing` / `vendor_prebuilt_stub_invalid` (invalid + gem not installed: refuse `vendor_prebuilt_stub_invalid` — no stub source exists) | refuse (`vendor_prebuilt_required` / `vendor_prebuilt_stub_invalid`) | | 401 / 403 grant / 5xx / network error | local build + `vendor_prebuilt_unavailable` | refuse | | `--offline` | local build | refuse (`vendor_service_offline_conflict`) | @@ -416,7 +417,24 @@ needs an eval-able stub gemspec that the `.gem` archive doesn't carry in bundler alongside the `.gem`, and the gem backend downloads + integrity-verifies both. A served gem whose stub is missing (a native-extension gem, for which the converter emits no stub, or a patch built before the stub rollout) is treated as a service miss — `auto` falls back to the local build, -`service` refuses (`vendor_prebuilt_required`). For any ecosystem with no service path at all +`service` refuses (`vendor_prebuilt_required`). A served stub that is present but INVALID — it +fails the rubygems `summary`/`authors` bar, so every bundler major would reject the vendored +path source at install time (a defect the 2026-08-19 live matrix found in every then-published +gem stub) — follows the same miss policy under its own code (additive/MINOR): `auto` falls back +to the local build with a loud `vendor_prebuilt_stub_invalid` warning naming the missing +attributes, `service` refuses with `vendor_prebuilt_stub_invalid`. (Semver note: before the +hardening, `service` mode exited 0 here while writing a stub bundler rejects — an UNINSTALLABLE +project. The refusal is the bug fix; the exit-0 was the defect, so this rides a MINOR.) When the +invalid-stub fallback finds the gem is ALSO not installed locally (no `specifications/` stub to +derive), the vendor refuses with the same `vendor_prebuilt_stub_invalid` code, naming the served +defect and the install-the-gem remedy. The locally-derived stub is validated at the same write +choke point: a corrupted local `specifications/` stub failing the bar refuses with +`gem_spec_invalid` naming the file. The bar is a conservative textual heuristic matched to what +rubygems 3.3–3.6 actually hard-fails (no assignment of `summary`; no `authors`/`author` +assignment, or one that collapses to no String elements — `[]`/`nil`/`[nil]`/`%w[]`; nil/empty +strings are rubygems-tolerated and pass); a valid stub is still written byte-verbatim, and the +idempotent re-vendor path re-checks the ON-DISK stub, routing a pre-hardening invalid one into +the artifact rebuild. For any ecosystem with no service path at all `auto`/`build` build locally as before, and `service` refuses with `vendor_service_unsupported_ecosystem`. A successful service vend emits `vendor_prebuilt_downloaded`. Unrelated to `--download-mode` (which selects the patch-CONTENT format for the local build). @@ -894,6 +912,8 @@ Every `--json` invocation emits a single JSON object that follows the **unified | `vendor_lock_checksums_unsupported` / `vendor_stale_lock_checksum` | `failed` | vendor (gem): an ambiguous/platform CHECKSUMS entry, or a v1-wired lock whose stale token blocks the hot path (run `vendor --revert` + re-vendor). | | `redirect_gem_stale_install` | `redirect.warnings[]` (warning) | scan `--mode hosted` (gem): a stale UNPATCHED materialization (installed gem, or committed `vendor/cache` archive) that `bundle install` will reuse instead of fetching the redirected patch; the detail carries the verified remedy. Full rules and flavors: the "Gem stale-install guard" section. | | `pypi_{poetry,pdm,pipenv}_no_lockfile` | `failed` | vendor (pypi): a lock-less tool marker with no `requirements.txt` fallback — run ` lock`. | +| `vendor_prebuilt_stub_invalid` | `failed` / `skipped` (warning) | vendor (gem, `--vendor-source`): the served stub gemspec fails the rubygems `summary`/`authors` bar, so bundler would refuse the vendored path source at install time. `service`: refusal naming the missing attributes; `auto`: loud warning + local-build fallback — or, when the gem is also not installed locally (no stub to derive), a refusal naming the served defect and the install-the-gem remedy. | +| `gem_spec_invalid` | `failed` | vendor (gem): the LOCAL `specifications/` stub gemspec fails the same rubygems `summary`/`authors` bar (a corrupted or hand-edited gem home); the refusal names the file — reinstall the gem (`gem pristine ` / fresh `bundle install`). | | `vendor_*` / `pypi_*` / `gemfile_*` / `lock_*` / `locked_version_mismatch` / `user_authored_*` / `native_extensions_unsupported` / `platform_gem_unsupported` | `failed`/`skipped` | vendor: per-ecosystem refusal + drift vocabulary; see the Vendor command contract section. New tags are additive (MINOR). | ### Top-level `EnvelopeError` codes diff --git a/crates/socket-patch-cli/tests/e2e_vendored_production.rs b/crates/socket-patch-cli/tests/e2e_vendored_production.rs index 41e0f036..2450bc12 100644 --- a/crates/socket-patch-cli/tests/e2e_vendored_production.rs +++ b/crates/socket-patch-cli/tests/e2e_vendored_production.rs @@ -52,22 +52,22 @@ //! | npm | `pkg:npm/minimist@1.2.2` | `80630680-4da6-45f9-bba8-b888e0ffd58c` | `Socket Community Patch` header | //! | PyPI | `pkg:pypi/urllib3@1.26.18` | *any of three* (see [`PYPI_UUIDS`]) | `Socket Community Patch` header | //! | Cargo | `pkg:cargo/traitobject@0.1.1` | `cf2e6f58-d9fa-4096-9151-c34afa717f89` | advisory id `GHSA-pp8r-vv2j-9j5v` | -//! | gem | `pkg:gem/activestorage@6.0.3` | `15e960b5-f432-4b6c-b8aa-534a2b419323` | `Socket Community Patch` header *(not yet asserted — see below)* | +//! | gem | `pkg:gem/activestorage@6.0.3` | `15e960b5-f432-4b6c-b8aa-534a2b419323` | `Socket Community Patch` header | //! -//! # Ecosystems with no full coverage, and why +//! # Ecosystem coverage notes (gaps, and one resolved gap) //! -//! * **gem** — the old `platform_gem_unsupported` refusal for -//! `?platform=ruby` purls was fixed in the CLI (#172 — only non-`ruby` -//! platform qualifiers are refused), and the 2026-08-18 catalog republish -//! restored the pinned patch, so the VENDOR itself now succeeds. -//! The full fresh-dir `bundle install` delivery proof is DEFERRED to the -//! stacked stub-hardening PR: the gem-stub-gemspec artifact production -//! serves is currently invalid (missing `summary`/`authors`), so rubygems -//! validation rejects the vendored `path:` source and `bundle install` -//! exits 1 on every bundler major (discovered 2026-08-19). -//! [`gem_bundler_vendored_known_platform_defect`] asserts redirect+download -//! hard and reports the vendor success. Promote with -//! `SOCKET_PATCH_VENDORED_E2E_GEM_STRICT=1`. +//! * **gem** — RESOLVED: full coverage. The old `platform_gem_unsupported` +//! refusal for `?platform=ruby` purls was fixed in the CLI (#172 — only +//! non-`ruby` platform qualifiers are refused), and the 2026-08-18 catalog +//! republish restored the pinned patch, so +//! [`gem_bundler_vendored_install_proof`] now runs the complete vendored +//! loop including the fresh-dir `bundle install` delivery proof. While +//! production's served `gem-stub-gemspec` remains invalid (D4: missing the +//! rubygems-required `summary`/`authors`), the leg passes via the CLI's +//! invalid-stub hardening — `--vendor-source auto` detects the defect and +//! falls back to the local build (`vendor_prebuilt_stub_invalid` warning); +//! once the server-side stub fix deploys and the artifacts rebuild, the +//! same leg exercises the service artifact directly. //! * **golang** — vendored mode *works* (directory `replace`), but production //! publishes no free golang patches, so there is nothing to vendor. //! [`golang_vendored_finds_no_free_patches`] asserts exactly that (zero @@ -341,9 +341,11 @@ fn scan_vendored(cwd: &Path, extra: &[&str]) -> serde_json::Value { /// /// `applied >= 1` is the anti-vacuity guard: a run that discovered nothing also /// exits 0 with `"status": "success"`, so without this a broken crawler would -/// look identical to a working vendor. `failed == 0` catches partial failures -/// (the gem leg deliberately does NOT go through here). The `applied` event's -/// purl may carry qualifiers (`?artifact_id=…`), so a substring match is used. +/// look identical to a working vendor. `failed == 0` catches partial failures. +/// The gem leg uses the purl-SCOPED [`assert_vendor_applied_for`] instead: its +/// fixture has many transitive gems, so run-wide counts would couple the leg +/// to the production catalog's future patches. The `applied` event's purl may +/// carry qualifiers (`?artifact_id=…`), so a substring match is used. fn assert_vendor_applied(env: &serde_json::Value, purl_needle: &str, leg: &str) { let vendor = &env["vendor"]; assert!( @@ -376,6 +378,52 @@ fn assert_vendor_applied(env: &serde_json::Value, purl_needle: &str, leg: &str) ); } +/// The events for purls containing `purl_needle` (substring — purls may carry +/// qualifiers) with the given `action`. +fn vendor_events_for<'a>( + env: &'a serde_json::Value, + purl_needle: &str, + action: &str, +) -> Vec<&'a serde_json::Value> { + env["vendor"]["events"] + .as_array() + .map(|events| { + events + .iter() + .filter(|e| { + e["action"] == action + && e["purl"] + .as_str() + .map(|p| p.contains(purl_needle)) + .unwrap_or(false) + }) + .collect() + }) + .unwrap_or_default() +} + +/// Purl-SCOPED variant of [`assert_vendor_applied`]: the named purl must have +/// an `applied` event and NO `failed` event. Other purls' outcomes are the +/// production catalog's business — a future free patch on a transitive gem of +/// the fixture must not red this leg — so run-wide `summary` counts are +/// deliberately not asserted. +fn assert_vendor_applied_for(env: &serde_json::Value, purl_needle: &str, leg: &str) { + assert!( + !env["vendor"].is_null(), + "{leg}: scan --mode vendored emitted no `vendor` sub-object — the CLI omits it \ + when discovery found nothing, so this means the crawler did not see the \ + installed dependency.\nenvelope:\n{env:#}" + ); + assert!( + !vendor_events_for(env, purl_needle, "applied").is_empty(), + "{leg}: no `applied` event for a purl containing `{purl_needle}`.\nenvelope:\n{env:#}" + ); + assert!( + vendor_events_for(env, purl_needle, "failed").is_empty(), + "{leg}: a `failed` event for `{purl_needle}`.\nenvelope:\n{env:#}" + ); +} + /// Assert the download phase resolved one of the expected patch UUIDs. fn assert_download_uuid(env: &serde_json::Value, uuids: &[&str], leg: &str) { let patches = env["download"]["patches"] @@ -424,6 +472,29 @@ fn vendor_revert(cwd: &Path, leg: &str) -> u64 { // Toolchain invocation // --------------------------------------------------------------------------- +/// Run `bundle ` with the ambient `BUNDLE_*`/`GEM_*`/`RUBYOPT` state +/// scrubbed first (a developer's global bundler config — frozen mode, a +/// custom BUNDLE_PATH or gem home, a RUBYOPT require — must not leak into a +/// leg that hard-asserts bundler outcomes; mirrors e2e_vendor_gem_build.rs). +/// The cache sandbox re-pins its own BUNDLE_USER_HOME/GEM_SPEC_CACHE after +/// the scrub, and the per-leg `env` is applied last so the leg's pins win. +fn bundle(cwd: &Path, args: &[&str], env: &[(&str, &str)]) -> Output { + let mut cmd = Command::new("bundle"); + cmd.args(args).current_dir(cwd); + for (k, _) in std::env::vars_os() { + let key = k.to_string_lossy().into_owned(); + if key.starts_with("BUNDLE_") || key.starts_with("GEM_") || key == "RUBYOPT" { + cmd.env_remove(&k); + } + } + cache_env::isolate(&mut cmd); + for (k, v) in env { + cmd.env(k, v); + } + cmd.output() + .unwrap_or_else(|e| panic!("failed to spawn `bundle`: {e}")) +} + /// Run an external package manager. Returns the `Output` without asserting, so /// callers can distinguish "the registry was unreachable" (soft-skip material /// during fixture setup) from "the install of the vendored lock failed" (always @@ -1670,38 +1741,68 @@ fn cargo_package_block(lock_text: &str, name: &str) -> Option { } // =========================================================================== -// RubyGems — vendor succeeds; full install proof deferred (invalid stub gemspec) +// RubyGems — bundler `path:` source // =========================================================================== -/// The gem redirect+download are asserted hard, and the vendor itself now -/// SUCCEEDS: #172 fixed the `platform_gem_unsupported` gate (only non-empty, -/// non-`ruby` platform qualifiers are refused) and the 2026-08-18 catalog -/// republish restored the pinned patch, so the success branch below — the -/// "appears FIXED" NOTE — is the live path today and the failure-tolerance -/// branch is vestigial. +/// Full vendored install proof for RubyGems: `scan --mode vendored` resolves +/// the free activestorage patch, materializes the patched gem under +/// `.socket/vendor/gem//-/`, and lands the mandatory +/// pair edit (Gemfile exact pin + `path:`, lock `PATH` section + +/// `(= )!` DEPENDENCIES pin). The delivery proof copies ONLY the +/// committable files into a fresh dir and runs a frozen `bundle install` +/// against a fresh empty `BUNDLE_PATH`: activestorage MUST resolve from the +/// vendored path source (its dependencies legitimately come from +/// rubygems.org — a path source only pins the one gem), and the file the +/// patch rewrites must carry the `Socket Community Patch` header. +/// +/// # How the leg passes while production's served stub is invalid (D4) +/// +/// The `gem-stub-gemspec` artifact production currently serves omits the +/// rubygems-required `summary`/`authors`, so writing it verbatim would make +/// the frozen `bundle install` below exit 1 on every bundler major. The CLI's +/// invalid-stub hardening is what this leg regression-tests live: under the +/// default `--vendor-source auto` the scan detects the defective stub, warns +/// (`vendor_prebuilt_stub_invalid`), and falls back to the LOCAL build +/// (installed gem + locally derived stub), which installs green. That is also +/// why the leg installs in bundler's deployment layout (`vendor/bundle` +/// inside the project): the crawler only sees a project-local install, and +/// the fallback needs the install's `specifications/` stub. Once the +/// server-side stub fix (depscan) deploys and the artifacts rebuild, the same +/// leg exercises the service artifact directly — no test change needed. /// -/// The upgrade to a full fresh-dir `bundle install` delivery proof is -/// DEFERRED to the stacked stub-hardening PR: the gem-stub-gemspec artifact -/// production serves is currently invalid (missing `summary`/`authors`), so -/// rubygems validation rejects the vendored `path:` source and -/// `bundle install` exits 1 on every bundler major (discovered 2026-08-19). -/// That PR carries the install proof as its regression test; until it lands, -/// `SOCKET_PATCH_VENDORED_E2E_GEM_STRICT=1` still promotes any vendor -/// failure here to a hard failure. +/// # History +/// +/// This leg used to tolerate a `platform_gem_unsupported` vendor refusal: +/// production publishes the gem purl platform-qualified (`?platform=ruby`) +/// and the old vendor gate refused every platform qualifier. #172 fixed the +/// gate to refuse only non-empty, non-`ruby` platforms, and the 2026-08-18 +/// catalog republish restored the pinned patch, so the leg was upgraded to +/// this full install proof per its own auto-retire NOTE. #[test] #[ignore = "live production API + real rubygems.org. Run with --ignored."] -fn gem_bundler_vendored_known_platform_defect() { - const LEG: &str = "gem_bundler_vendored_known_platform_defect"; +fn gem_bundler_vendored_install_proof() { + const LEG: &str = "gem_bundler_vendored_install_proof"; if !has_command("ruby") || !has_command("bundle") { soft_skip!(LEG, "`ruby` and/or `bundle` not on PATH"); } let tmp = tempfile::tempdir().expect("tempdir"); let proj = tmp.path().join("proj"); std::fs::create_dir_all(&proj).expect("mkdir proj"); - let bundle_path = tmp.path().join("bundle").display().to_string(); + // Deployment layout — `vendor/bundle` INSIDE the project: the gem crawler + // probes `vendor/bundle//*/gems/` in local mode, and the vendor + // backend's local-build fallback needs the crawler-visible install (the + // `specifications/` sibling carries the local stub gemspec) while + // production's served stub is invalid (D4). The app config stays OUTSIDE + // the project so no `.bundle/config` joins the committable set. + let bundle_path = proj.join("vendor/bundle").display().to_string(); + let bundle_config = tmp.path().join("bundle-config").display().to_string(); + // mimemagic (pulled via activestorage → marcel) builds against the system + // shared-mime-info DB, which not every host installs — use the gem's + // bundled placeholder instead of depending on a host package. let env = [ ("BUNDLE_PATH", bundle_path.as_str()), - ("BUNDLE_APP_CONFIG", bundle_path.as_str()), + ("BUNDLE_APP_CONFIG", bundle_config.as_str()), + ("USE_FREEDESKTOP_PLACEHOLDER", "true"), ]; std::fs::write( @@ -1710,85 +1811,204 @@ fn gem_bundler_vendored_known_platform_defect() { ) .expect("write Gemfile"); - if !ok(&tool(&proj, "bundle", &["lock"], &env)) { + if !ok(&bundle(&proj, &["lock"], &env)) { soft_skip!(LEG, "`bundle lock` failed"); } - let install = tool(&proj, "bundle", &["install", "--quiet"], &env); + let install = bundle(&proj, &["install", "--quiet"], &env); if !ok(&install) { soft_skip!(LEG, "upstream `bundle install` failed:\n{}", dump(&install)); } - // Raw invocation: this run is EXPECTED to exit non-zero on the known defect. - let (code, stdout, stderr) = run_socket( - &proj, - &[ - "scan", - "--json", - "--yes", - "--mode", - "vendored", - "--cwd", - proj.to_str().unwrap(), - ], + // Anti-vacuity: the upstream install must be pristine. `bundle info + // --path` reports the exact directory bundler resolved for the gem. + let info = bundle(&proj, &["info", GEM_NAME, "--path"], &env); + assert!( + ok(&info), + "{LEG}: `bundle info {GEM_NAME} --path` failed after the upstream install:\n{}", + dump(&info) ); - let env_json: serde_json::Value = serde_json::from_str(stdout.trim()).unwrap_or_else(|e| { - panic!("{LEG}: scan --mode vendored did not emit JSON ({e}).\nstdout:\n{stdout}\nstderr:\n{stderr}") - }); + let installed_dir = PathBuf::from(String::from_utf8_lossy(&info.stdout).trim()); + let patched_file_rel = "lib/active_storage/service/s3_service.rb"; + assert_pristine(&installed_dir.join(patched_file_rel), PATCH_MARKER, LEG); + let pristine = std::fs::read(installed_dir.join(patched_file_rel)).unwrap(); + let gemfile_before = std::fs::read(proj.join("Gemfile")).unwrap(); + let lock_before = std::fs::read(proj.join("Gemfile.lock")).unwrap(); - // The download must have succeeded regardless — that half is not defective. + let env_json = scan_vendored(&proj, &[]); assert_download_uuid(&env_json, &[GEM_UUID], LEG); - let dl_failed = env_json["download"]["failed"].as_u64().unwrap_or(0); assert_eq!( - dl_failed, 0, - "{LEG}: the gem patch download itself failed — that is a regression, not the known \ - vendor-platform defect.\nenvelope:\n{env_json:#}" + env_json["download"]["failed"].as_u64().unwrap_or(99), + 0, + "{LEG}: the gem patch download failed.\nenvelope:\n{env_json:#}" + ); + // Purl-scoped (NOT run-wide counts): a future free patch on one of the + // fixture's transitive Rails gems must not red this leg. + assert_vendor_applied_for(&env_json, &format!("{GEM_NAME}@{GEM_VERSION}"), LEG); + + // Route attribution: EXACTLY one of the two markers must be present for + // this purl — the service artifact was used (`vendor_prebuilt_downloaded`) + // or the invalid-served-stub fallback built locally + // (`vendor_prebuilt_stub_invalid`). Asserting exactly one auto-retires the + // fallback expectation the moment the depscan stub fix deploys and the + // rebuilt artifacts serve valid stubs (and catches both-or-neither as a + // defect either way). + let route_markers = env_json["vendor"]["events"] + .as_array() + .cloned() + .unwrap_or_default() + .iter() + .filter(|e| { + e["purl"] + .as_str() + .map(|p| p.contains(GEM_NAME)) + .unwrap_or(false) + && matches!( + e["errorCode"].as_str(), + Some("vendor_prebuilt_stub_invalid") | Some("vendor_prebuilt_downloaded") + ) + }) + .count(); + assert_eq!( + route_markers, 1, + "{LEG}: expected exactly one route marker (vendor_prebuilt_downloaded XOR \ + vendor_prebuilt_stub_invalid) for {GEM_NAME}.\nenvelope:\n{env_json:#}" ); - let gem_strict = std::env::var("SOCKET_PATCH_VENDORED_E2E_GEM_STRICT") - .map(|v| v == "1" || v.eq_ignore_ascii_case("true")) - .unwrap_or(false); + // The committable artifact + the mandatory pair edit. + let copy_rel = format!(".socket/vendor/gem/{GEM_UUID}/{GEM_NAME}-{GEM_VERSION}"); + assert_patched( + &proj + .join(©_rel) + .join("lib/active_storage/service/s3_service.rb"), + PATCH_MARKER, + LEG, + ); + let gemfile = read(&proj.join("Gemfile")); + assert!( + gemfile.contains(&format!( + "gem \"{GEM_NAME}\", \"{GEM_VERSION}\", path: \"{copy_rel}\"" + )), + "{LEG}: Gemfile line not rewritten to the exact-pin + path: form:\n{gemfile}" + ); + let lock = read(&proj.join("Gemfile.lock")); + assert!( + lock.contains(&format!( + "PATH\n remote: {copy_rel}\n specs:\n {GEM_NAME} ({GEM_VERSION})" + )), + "{LEG}: canonical PATH section missing from Gemfile.lock:\n{lock}" + ); + assert!( + lock.contains(&format!("\n {GEM_NAME} (= {GEM_VERSION})!")), + "{LEG}: DEPENDENCIES pin ` {GEM_NAME} (= {GEM_VERSION})!` missing from Gemfile.lock:\n{lock}" + ); + + // DELIVERY PROOF: ONLY the committable files (Gemfile, Gemfile.lock, + // .socket/ — the leg's BUNDLE_APP_CONFIG lives outside the project, so + // there is no .bundle/config to commit), a fresh EMPTY BUNDLE_PATH, and + // a frozen `bundle install`. Frozen mode makes bundler enforce the + // committed lock — including its vendored PATH source — and fail rather + // than re-resolve, mirroring docker_e2e_vendor_gem's stage 2. The gem's + // dependencies come from rubygems.org (a path source pins only the one + // gem), so the install is online; the point is that activestorage itself + // can only come from `.socket/vendor/`. + let fresh = tmp.path().join("fresh"); + std::fs::create_dir_all(&fresh).unwrap(); + std::fs::copy(proj.join("Gemfile"), fresh.join("Gemfile")).unwrap(); + std::fs::copy(proj.join("Gemfile.lock"), fresh.join("Gemfile.lock")).unwrap(); + copy_dir_recursive(&proj.join(".socket"), &fresh.join(".socket")); + let fresh_bundle = tmp.path().join("fresh-bundle").display().to_string(); + let fresh_env = [ + ("BUNDLE_PATH", fresh_bundle.as_str()), + ("BUNDLE_APP_CONFIG", fresh_bundle.as_str()), + ("BUNDLE_FROZEN", "true"), + // Same shared-mime-info hazard as the upstream install above. + ("USE_FREEDESKTOP_PLACEHOLDER", "true"), + ]; + let lock_committed = std::fs::read(fresh.join("Gemfile.lock")).unwrap(); + let fresh_install = bundle(&fresh, &["install"], &fresh_env); + assert!( + ok(&fresh_install), + "{LEG}: frozen `bundle install` from the committable files failed:\n{}", + dump(&fresh_install) + ); + assert_eq!( + std::fs::read(fresh.join("Gemfile.lock")).unwrap(), + lock_committed, + "{LEG}: frozen `bundle install` churned the committed Gemfile.lock" + ); + // Bundler must have resolved the gem FROM the vendored path source INSIDE + // the fresh dir, and the bytes it will load must carry the patch marker + // and differ from the captured pristine registry bytes. (`contains` + // alone would also match the ORIGINAL project's vendored path; + // canonicalize both sides — macOS reports tempdirs via /var symlinked to + // /private/var.) + let fresh_info = bundle(&fresh, &["info", GEM_NAME, "--path"], &fresh_env); + assert!( + ok(&fresh_info), + "{LEG}: `bundle info {GEM_NAME} --path` failed after the fresh install:\n{}", + dump(&fresh_info) + ); + let resolved = String::from_utf8_lossy(&fresh_info.stdout) + .trim() + .to_string(); + let resolved_canon = std::fs::canonicalize(&resolved) + .unwrap_or_else(|e| panic!("{LEG}: cannot canonicalize `{resolved}`: {e}")); + let fresh_canon = std::fs::canonicalize(&fresh).unwrap(); + assert!( + resolved_canon.starts_with(&fresh_canon) && resolved.contains(©_rel), + "{LEG}: bundler resolved {GEM_NAME} from `{resolved}`, not the vendored \ + path `{copy_rel}` inside the fresh dir — the pair edit did not take \ + effect in the fresh dir" + ); + assert_patched( + &PathBuf::from(&resolved).join(patched_file_rel), + PATCH_MARKER, + LEG, + ); + assert_ne!( + std::fs::read(PathBuf::from(&resolved).join(patched_file_rel)).unwrap(), + pristine, + "{LEG}: the reinstalled bytes equal the PRISTINE registry bytes — the vendored \ + artifact was not the one installed" + ); - let applied = env_json["vendor"]["summary"]["applied"] - .as_u64() - .unwrap_or(0); - if code == 0 && applied >= 1 { - // The CLI now vendors platform gems. Say so loudly — this leg should be - // promoted to a full delivery proof (bundle install frozen) and this - // tolerance branch deleted. - println!( - "NOTE {LEG}: `scan --mode vendored` now SUCCEEDS for {GEM_PURL} (applied={applied}). \ - The platform_gem_unsupported vendor gap appears FIXED — upgrade this leg to a full \ - fresh-checkout `bundle install` delivery proof and delete the tolerance branch." - ); - return; - } + // Idempotency + revert (mirrors the pip/uv legs; purl-scoped so a future + // free patch on a transitive gem cannot red the leg). + let gemfile_wired = std::fs::read(proj.join("Gemfile")).unwrap(); + let lock_wired = std::fs::read(proj.join("Gemfile.lock")).unwrap(); + let env2 = scan_vendored(&proj, &[]); + assert!( + vendor_events_for(&env2, GEM_NAME, "applied").is_empty(), + "{LEG}: re-run must vendor nothing new for {GEM_NAME}:\n{env2:#}" + ); + assert_eq!( + std::fs::read(proj.join("Gemfile")).unwrap(), + gemfile_wired, + "{LEG}: re-run must leave the Gemfile byte-identical" + ); + assert_eq!( + std::fs::read(proj.join("Gemfile.lock")).unwrap(), + lock_wired, + "{LEG}: re-run must leave Gemfile.lock byte-identical" + ); - // Otherwise: it must be exactly the known `platform_gem_unsupported` failure. - let events = env_json["vendor"]["events"] - .as_array() - .cloned() - .unwrap_or_default(); - let is_known = events - .iter() - .any(|e| e["action"] == "failed" && e["errorCode"] == "platform_gem_unsupported"); assert!( - !gem_strict, - "{LEG}: SOCKET_PATCH_VENDORED_E2E_GEM_STRICT=1 and vendoring {GEM_PURL} did not succeed \ - (exit {code}).\nenvelope:\n{env_json:#}\nstderr:\n{stderr}" + vendor_revert(&proj, LEG) >= 1, + "{LEG}: at least the {GEM_NAME} entry reverted" + ); + assert_eq!( + std::fs::read(proj.join("Gemfile")).unwrap(), + gemfile_before, + "{LEG}: revert must restore the Gemfile byte-identical" + ); + assert_eq!( + std::fs::read(proj.join("Gemfile.lock")).unwrap(), + lock_before, + "{LEG}: revert must restore Gemfile.lock byte-identical" ); assert!( - is_known, - "{LEG}: `scan --mode vendored` failed for {GEM_PURL}, but NOT with the known \ - `platform_gem_unsupported` vendor code — this is a new regression (exit {code}).\n\ - envelope:\n{env_json:#}\nstderr:\n{stderr}" - ); - println!( - "KNOWN CLI GAP {LEG}: `scan --mode vendored` downloads the {GEM_NAME} patch but the \ - vendor backend refuses the platform-qualified purl \ - (pkg:gem/{GEM_NAME}@{GEM_VERSION}?platform=ruby) with `platform_gem_unsupported`. \ - Redirect+download asserted; the delivery proof is blocked until the CLI learns to \ - vendor platform gems. This leg starts asserting a real install automatically once the \ - vendor succeeds." + !proj.join(".socket/vendor").exists(), + "{LEG}: .socket/vendor must be gone after revert" ); } diff --git a/crates/socket-patch-cli/tests/in_process_vendor.rs b/crates/socket-patch-cli/tests/in_process_vendor.rs index a07ff897..e54baf87 100644 --- a/crates/socket-patch-cli/tests/in_process_vendor.rs +++ b/crates/socket-patch-cli/tests/in_process_vendor.rs @@ -1734,7 +1734,7 @@ const GEM_PURL: &str = "pkg:gem/demo-gem@1.0.0"; const GEM_PURL_QUALIFIED: &str = "pkg:gem/demo-gem@1.0.0?platform=ruby"; const GEM_ORIG: &[u8] = b"module DemoGem\n STATUS = \"orig\"\nend\n"; const GEM_PATCHED: &[u8] = b"module DemoGem\n STATUS = \"patched\"\nend\n"; -const GEM_GEMSPEC: &str = "Gem::Specification.new do |s|\n s.name = \"demo-gem\"\n s.version = \"1.0.0\"\n s.summary = \"in-process scan --vendor fixture\"\n s.require_paths = [\"lib\"]\nend\n"; +const GEM_GEMSPEC: &str = "Gem::Specification.new do |s|\n s.name = \"demo-gem\"\n s.version = \"1.0.0\"\n s.summary = \"in-process scan --vendor fixture\"\n s.authors = [\"socket-patch e2e\"]\n s.require_paths = [\"lib\"]\nend\n"; const GEM_GEMFILE: &str = "source \"https://rubygems.org\"\n\ngem \"demo-gem\", \"~> 1.0\"\n"; /// Hand-pinned bundler lock grammar (no CHECKSUMS — the 2.x/3.x default). const GEM_LOCK: &str = "GEM\n remote: https://rubygems.org/\n specs:\n demo-gem (1.0.0)\n\nPLATFORMS\n ruby\n\nDEPENDENCIES\n demo-gem (~> 1.0)\n\nBUNDLED WITH\n 2.6.2\n"; diff --git a/crates/socket-patch-cli/tests/repair_vendor_e2e.rs b/crates/socket-patch-cli/tests/repair_vendor_e2e.rs index 41a5d869..faab137f 100644 --- a/crates/socket-patch-cli/tests/repair_vendor_e2e.rs +++ b/crates/socket-patch-cli/tests/repair_vendor_e2e.rs @@ -1150,7 +1150,10 @@ const GEM_NAME: &str = "padlock"; const GEM_VERSION: &str = "1.2.0"; const GEM_PURL: &str = "pkg:gem/padlock@1.2.0"; const GEM_ENCODED: &str = "pkg%3Agem%2Fpadlock%401.2.0"; -const GEMSPEC_STUB: &[u8] = b"Gem::Specification.new do |s|\n s.name = \"padlock\"\n s.version = \"1.2.0\"\n s.require_paths = [\"lib\"]\nend\n"; +// Assigns the rubygems-required `summary` + `authors` (as every healthy +// rubygems-written stub does): the vendor/rebuild write choke point validates +// them since the D4 invalid-stub hardening. +const GEMSPEC_STUB: &[u8] = b"Gem::Specification.new do |s|\n s.name = \"padlock\"\n s.version = \"1.2.0\"\n s.summary = \"repair fixture\"\n s.authors = [\"socket-patch e2e\"]\n s.require_paths = [\"lib\"]\nend\n"; fn gem_copy_rel() -> String { format!(".socket/vendor/gem/{GEM_UUID}/{GEM_NAME}-{GEM_VERSION}") @@ -1811,7 +1814,7 @@ async fn repair_gem_dir_tamper_matrix_and_vex_refusal() { /// repair loops the same failure. #[tokio::test] async fn repair_refreshes_stale_inventory_from_service_provenance() { - const SERVICE_STUB: &[u8] = b"# converter-generated stub\nGem::Specification.new do |s|\n s.name = \"padlock\"\n s.version = \"1.2.0\"\n s.require_paths = [\"lib\"]\nend\n"; + const SERVICE_STUB: &[u8] = b"# converter-generated stub\nGem::Specification.new do |s|\n s.name = \"padlock\"\n s.version = \"1.2.0\"\n s.summary = \"repair fixture\"\n s.authors = [\"socket-patch e2e\"]\n s.require_paths = [\"lib\"]\nend\n"; let mock = MockServer::start().await; mount_gem_patch_api(&mock).await; let tmp = tempfile::tempdir().unwrap(); diff --git a/crates/socket-patch-core/src/vendor/gem.rs b/crates/socket-patch-core/src/vendor/gem.rs index de369999..46dcb29a 100644 --- a/crates/socket-patch-core/src/vendor/gem.rs +++ b/crates/socket-patch-core/src/vendor/gem.rs @@ -51,7 +51,7 @@ //! sources and the missing `.so` only fails at `require` time with a //! confusing error — refusing up front is the honest failure. -use std::path::Path; +use std::path::{Path, PathBuf}; use serde_json::Value; @@ -254,20 +254,32 @@ pub async fn vendor_gem( }; // ── stub gemspec (local) ───────────────────────────────────────────── - // `specifications/` is a sibling of `gems/`; derive it from installed_dir. + // `specifications/` is a sibling of `gems/`; derive it from installed_dir + // ONLY when installed_dir actually sits inside a gem home's `gems/` dir. + // SECURITY: the registry auto-fetch ladder stages a not-installed gem at + // `/-` (registry_fetch::fetch_gem); + // walking two parents up from THERE escapes the private dir into the + // SHARED temp root, making `$TMPDIR/specifications/.gemspec` a + // predictable, attacker-plantable path on multi-user hosts — one whose + // contents would be committed into the project and later eval'd as Ruby + // by every `bundle install`. A staging dir has no local stub, period. + // // The read is non-fatal: the LOCAL build needs this stub, but the service // path brings its own (the converter-generated `gem-stub-gemspec`), so an // auto-fetched (not-installed) gem whose only `installed_dir` is a bare // `data.tar.gz` extraction can still vendor via the service. The // `gem_spec_missing` refusal moves into the local-build fallback, where the // stub is actually required. - let spec_src = installed_dir - .parent() - .and_then(Path::parent) - .map(|home| home.join("specifications").join(format!("{leaf}.gemspec"))); - let spec_text: Option = match &spec_src { - Some(p) => tokio::fs::read_to_string(p).await.ok(), - None => None, + let local_stub: Option<(PathBuf, String)> = { + let spec_src = installed_dir + .parent() + .filter(|gems| gems.file_name().is_some_and(|n| n == "gems")) + .and_then(Path::parent) + .map(|home| home.join("specifications").join(format!("{leaf}.gemspec"))); + match spec_src { + Some(p) => tokio::fs::read_to_string(&p).await.ok().map(|t| (p, t)), + None => None, + } }; // Textual heuristic, deliberately fail-closed on a match: bundler skips // extension builds for path sources entirely, so a native gem would @@ -275,7 +287,7 @@ pub async fn vendor_gem( // Only the local stub is checked here (when present); the service stub is // re-checked in `gem_service_copy`, and a native gem emits no service stub // at all (the converter refuses it), so the service path also misses. - if let Some(text) = &spec_text { + if let Some((_, text)) = &local_stub { if gemspec_declares_extensions(text) { return refused( "native_extensions_unsupported", @@ -294,10 +306,17 @@ pub async fn vendor_gem( let remote_line = format!(" remote: {copy_rel}"); let lock_wired = lock_text.split('\n').any(|l| l == remote_line) && gemfile_text.contains(©_rel); - let copy_ok = copy_matches_after_hashes(©_dir, &record.files).await - && tokio::fs::metadata(copy_dir.join(format!("{name}.gemspec"))) - .await - .is_ok(); + // D4 heal: a project vendored before the invalid-stub hardening carries + // the defective SERVED stub on disk, so EXISTS is not enough — an on-disk + // stub that fails the required-attribute bar routes into the artifact + // rebuild below (which re-materialises a valid stub) instead of the + // silent `already_vendored` no-op. + let copy_stub_ok = + match tokio::fs::read_to_string(copy_dir.join(format!("{name}.gemspec"))).await { + Ok(text) => gemspec_missing_required_attrs(&text).is_empty(), + Err(_) => false, + }; + let copy_ok = copy_matches_after_hashes(©_dir, &record.files).await && copy_stub_ok; if lock_wired { if lock_checksum_in_sync(&lock_text, name, version) { if copy_ok { @@ -325,7 +344,7 @@ pub async fn vendor_gem( &uuid_dir, name, version, - spec_text.as_deref(), + local_stub.as_ref().map(|(p, t)| (p.as_path(), t.as_str())), record, sources, force, @@ -408,7 +427,7 @@ pub async fn vendor_gem( &uuid_dir, name, version, - spec_text.as_deref(), + local_stub.as_ref().map(|(p, t)| (p.as_path(), t.as_str())), record, sources, force, @@ -652,7 +671,13 @@ enum GemServiceCopy { /// Bubble this terminal outcome (boxed — `VendorOutcome` is large). HardFail(Box), /// Fall back to copying the installed gem + local stub and patching it. - FallBack, + /// When the service DID serve a stub but it failed validation (the D4 + /// defect), the payload carries the defect reason so a stub-less local + /// fallback can refuse truthfully — naming the served defect and the + /// install-the-gem remedy — instead of `gem_spec_missing`'s circular + /// "use --vendor-source=service" advice (a `Refused` outcome carries no + /// warnings, so without this the diagnostic never reaches the envelope). + FallBack(Option), } /// Download the prebuilt `.gem` + its `gem-stub-gemspec` secondary artifact, @@ -667,7 +692,10 @@ enum GemServiceCopy { /// emits no stub — bundler can't build extensions for a path source) or a gem /// patch built before the stub rollout (the invalidation migration rebuilds /// those). The downloaded stub is re-checked for native extensions as defense -/// in depth. +/// in depth, and an INVALID stub — one missing the rubygems-required +/// `summary`/`authors` assignments, the D4 served-artifact defect — follows +/// the same miss policy under its own `vendor_prebuilt_stub_invalid` code +/// (always loud, even under `auto`). async fn gem_service_copy( service: Option<&VendorServiceConfig>, record: &PatchRecord, @@ -677,23 +705,39 @@ async fn gem_service_copy( warnings: &mut Vec, ) -> GemServiceCopy { let Some(cfg) = service else { - return GemServiceCopy::FallBack; + return GemServiceCopy::FallBack(None); }; if !cfg.service_enabled() { - return GemServiceCopy::FallBack; + return GemServiceCopy::FallBack(None); } fn hard(code: &'static str, detail: String) -> GemServiceCopy { GemServiceCopy::HardFail(Box::new(refused(code, detail))) } - let miss = |warnings: &mut Vec, code: &'static str, reason: String| { + // One policy for every service miss: explicit `service` refuses (the + // `refusal` tuple names the terminal code and an optional remedy sentence + // for its detail), `auto` warns under `code` and falls back to the local + // build. `is_stub_defect` marks the misses where the service DID serve a + // stub that failed validation — the reason then rides the `FallBack` + // payload (see [`GemServiceCopy::FallBack`]). + let miss = |warnings: &mut Vec, + code: &'static str, + refusal: (&'static str, &str), + reason: String, + is_stub_defect: bool| { if cfg.source.requires_service() { - hard("vendor_prebuilt_required", reason) + let (hard_code, remedy) = refusal; + let detail = if remedy.is_empty() { + reason + } else { + format!("{reason}. {remedy}") + }; + hard(hard_code, detail) } else { warnings.push(VendorWarning::new( code, format!("{reason}; building locally instead"), )); - GemServiceCopy::FallBack + GemServiceCopy::FallBack(is_stub_defect.then_some(reason)) } }; @@ -704,14 +748,18 @@ async fn gem_service_copy( return miss( warnings, "vendor_prebuilt_integrity_mismatch", + ("vendor_prebuilt_required", ""), format!("prebuilt .gem failed integrity ({reason})"), + false, ); } ServiceArtifact::Pending => { return miss( warnings, "vendor_prebuilt_pending", + ("vendor_prebuilt_required", ""), "prebuilt .gem is still building".to_string(), + false, ); } ServiceArtifact::Unavailable(reason) => { @@ -721,13 +769,15 @@ async fn gem_service_copy( format!("prebuilt .gem unavailable: {reason}"), ); } - return GemServiceCopy::FallBack; + return GemServiceCopy::FallBack(None); } ServiceArtifact::Failed(reason) => { return miss( warnings, "vendor_prebuilt_unavailable", + ("vendor_prebuilt_required", ""), format!("patch service request failed ({reason})"), + false, ); } }; @@ -739,31 +789,38 @@ async fn gem_service_copy( return miss( warnings, "vendor_prebuilt_stub_missing", + ("vendor_prebuilt_required", ""), "the patch service served no stub gemspec for this gem (a native-extension \ gem, or a patch built before the stub rollout)" .to_string(), + false, ); } SecondaryArtifactResult::IntegrityMismatch(reason) => { return miss( warnings, "vendor_prebuilt_integrity_mismatch", + ("vendor_prebuilt_required", ""), format!("prebuilt stub gemspec failed integrity ({reason})"), + false, ); } SecondaryArtifactResult::Failed(reason) => { return miss( warnings, "vendor_prebuilt_unavailable", + ("vendor_prebuilt_required", ""), format!("could not fetch the stub gemspec ({reason})"), + false, ); } }; + let stub_text = String::from_utf8_lossy(&stub); // Defense in depth: the converter does not emit a stub for native gems, but // refuse one here too — bundler silently skips extension builds for path // sources, so a native gem would install and then fail at `require` time. - if gemspec_declares_extensions(&String::from_utf8_lossy(&stub)) { + if gemspec_declares_extensions(&stub_text) { return hard( "native_extensions_unsupported", format!( @@ -773,6 +830,42 @@ async fn gem_service_copy( ); } + // Defense in depth (D4, gem live matrix 2026-08-19): production's stub + // generator omitted the rubygems-required `summary`/`authors`, and every + // bundler major validates path-source gemspecs — writing such a stub + // verbatim makes every later `bundle install` exit 1 (`missing value for + // attribute summary`). An INVALID stub follows the MISSING-stub policy + // (fall back under `auto`, refuse under `service`) but under its own + // `vendor_prebuilt_stub_invalid` code, and always loudly — the served + // artifact is defective, not merely absent. Nothing has been written yet, + // so the refusal leaves no partial artifacts. + let missing_attrs = gemspec_missing_required_attrs(&stub_text); + if !missing_attrs.is_empty() { + let licenses_note = if gemspec_assigns_attr(&stub_text, &["licenses", "license"]) { + "" + } else { + " (it also omits `licenses`, a rubygems warning)" + }; + let reason = format!( + "the served stub gemspec for {name} is invalid: it does not assign the \ + rubygems-required attribute(s) {}{licenses_note}; bundler validates \ + path-source gemspecs, so vendoring it would make every later \ + `bundle install` fail", + missing_attrs.join(", "), + ); + return miss( + warnings, + "vendor_prebuilt_stub_invalid", + ( + "vendor_prebuilt_stub_invalid", + "Re-run with --vendor-source=auto (or build) to vendor from the locally \ + installed gem until the service artifact is rebuilt", + ), + reason, + true, + ); + } + // Extract the patched `.gem`'s data.tar.gz into a clean copy dir, then add // the stub as `.gemspec` (a `.gem`'s data.tar.gz never carries one — // the gemspec lives in metadata.gz). @@ -809,10 +902,12 @@ async fn gem_service_copy( return miss( warnings, "vendor_prebuilt_layout_mismatch", + ("vendor_prebuilt_required", ""), format!( "prebuilt .gem for {name} extracted to an unexpected layout \ (patched files absent at their recorded paths)" ), + false, ); } warnings.push(VendorWarning::new( @@ -840,7 +935,7 @@ async fn materialise_patched_copy( uuid_dir: &Path, name: &str, version: &str, - spec_text: Option<&str>, + local_stub: Option<(&Path, &str)>, record: &PatchRecord, sources: &PatchSources<'_>, force: bool, @@ -854,19 +949,63 @@ async fn materialise_patched_copy( Ok(already_patched_result(purl, copy_dir, &record.files)) } GemServiceCopy::HardFail(outcome) => Err(outcome), - GemServiceCopy::FallBack => { + GemServiceCopy::FallBack(served_stub_defect) => { // The local build needs the stub gemspec from the installed gem's // `specifications/` dir — absent for an auto-fetched (not-installed) // gem, whose only route is the service path. - let Some(spec_text) = spec_text else { + let Some((spec_path, spec_text)) = local_stub else { + return Err(Box::new(match served_stub_defect { + // The service DID serve a stub — a defective one (D4). Say + // so: the generic advice below would send the user in a + // circle (`--vendor-source=service` refuses on the same + // defect), and a `Refused` outcome carries no warnings, so + // this detail is the diagnostic's only route into the + // envelope. + Some(defect) => refused( + "vendor_prebuilt_stub_invalid", + format!( + "{defect}; and {name}@{version} is not installed locally, so the \ + local-build fallback has no stub gemspec to derive from — install \ + the gem (e.g. `bundle install`) and re-run, or wait for the \ + rebuilt service artifact" + ), + ), + None => refused( + "gem_spec_missing", + format!( + "no local stub gemspec for {name}@{version} (a path source cannot \ + be wired without one); install the gem or use \ + --vendor-source=service" + ), + ), + })); + }; + // The write choke point validates BOTH stub sources: the served + // stub is checked in `gem_service_copy`, and the locally-derived + // stub here — bundler rejects a path-source gemspec missing the + // required attributes wherever it came from. (A healthy rubygems + // install always writes a valid `specifications/` stub, so this + // only fires on a corrupted or hand-edited gem home.) + let missing = gemspec_missing_required_attrs(spec_text); + if !missing.is_empty() { + let served_note = match served_stub_defect { + Some(defect) => { + format!("; the patch service cannot supply one either ({defect})") + } + None => String::new(), + }; return Err(Box::new(refused( - "gem_spec_missing", + "gem_spec_invalid", format!( - "no local stub gemspec for {name}@{version} (a path source cannot be \ - wired without one); install the gem or use --vendor-source=service" + "the local stub gemspec at {} does not assign the rubygems-required \ + attribute(s) {} — bundler would refuse the vendored path source at \ + install time; reinstall the gem (`gem pristine {name}` or a fresh \ + `bundle install`) and re-run{served_note}", + spec_path.display(), + missing.join(", "), ), ))); - }; + } if let Err(e) = fresh_copy(installed_dir, copy_dir, None).await { return Ok(synthesized_result( purl, @@ -2170,16 +2309,53 @@ fn is_plain_gem_token(s: &str) -> bool { .all(|c| c.is_ascii_alphanumeric() || matches!(c, '.' | '_' | '-')) } +/// The one shared gemspec line-scanner: locate a `.{attr}` mention in `line` +/// and return what follows it (leading-whitespace-trimmed), or `None`. +/// +/// `anchored` additionally requires the mention to OPEN the line as +/// `.{attr}` with a plain-identifier receiver (`s.summary = …`, +/// ` spec.authors= …`). Anchoring makes a preceding comment marker +/// impossible, so anchored callers scan RAW lines with no comment-stripping — +/// stripping at `#` would truncate inside string literals and misjudge +/// `s.summary = "#1 Ruby web server"` as missing. A mention whose attr +/// continues as a longer identifier (`.extensions_dir`, `.authors` when +/// looking for `.author`) is never a match. Only the FIRST mention per line +/// is examined — one attribute per line is the shape `Specification#to_ruby` +/// emits. Parsing ruby for real would need a ruby. +fn attr_mention<'a>(line: &'a str, attr: &str, anchored: bool) -> Option<&'a str> { + let needle = format!(".{attr}"); + let idx = line.find(&needle)?; + if anchored { + let receiver = line[..idx].trim_start(); + if receiver.is_empty() + || !receiver + .chars() + .all(|c| c.is_ascii_alphanumeric() || matches!(c, '_' | '@')) + { + return None; + } + } + let after = &line[idx + needle.len()..]; + if after + .chars() + .next() + .is_some_and(|c| c.is_ascii_alphanumeric() || c == '_') + { + return None; + } + Some(after.trim_start()) +} + /// Textual heuristic for `s.extensions = […]` / `spec.extensions << …` style -/// declarations (comment-stripped per line). A match always refuses -/// (fail-closed); a miss — e.g. extensions assigned through interpolation -/// tricks — falls through, which only loses the refusal's nicer error, not -/// safety. Parsing ruby for real would need a ruby. +/// declarations (comment-stripped per line — a commented-out declaration is +/// not one; the truncation caveat in [`attr_mention`] only loses this +/// refusal's nicer error, never safety, because a match REFUSES). A miss — +/// e.g. extensions assigned through interpolation tricks — falls through, +/// which likewise only loses the nicer error. fn gemspec_declares_extensions(spec_text: &str) -> bool { for raw in spec_text.lines() { let line = raw.split('#').next().unwrap_or(""); - if let Some(idx) = line.find(".extensions") { - let after = line[idx + ".extensions".len()..].trim_start(); + if let Some(after) = attr_mention(line, "extensions", false) { if (after.starts_with('=') && !after.starts_with("==")) || after.starts_with("<<") || after.starts_with("+=") @@ -2193,6 +2369,88 @@ fn gemspec_declares_extensions(spec_text: &str) -> bool { false } +/// Every RHS assigned to any of the `attrs` aliases at a line start +/// (assignments only — `==` comparisons don't count), via [`attr_mention`]. +fn gemspec_attr_rhs<'a>(spec_text: &'a str, attrs: &[&str]) -> Vec<&'a str> { + let mut out = Vec::new(); + for raw in spec_text.lines() { + for attr in attrs { + if let Some(after) = attr_mention(raw, attr, true) { + if let Some(rhs) = after.strip_prefix('=') { + if !rhs.starts_with('=') { + out.push(rhs.trim()); + } + } + } + } + } + out +} + +/// Does any line assign one of the `attrs` aliases? Pass every alias rubygems +/// accepts for the attribute (`["authors", "author"]`, `["licenses", +/// "license"]`). +fn gemspec_assigns_attr(spec_text: &str, attrs: &[&str]) -> bool { + !gemspec_attr_rhs(spec_text, attrs).is_empty() +} + +/// Textually: does this `authors` RHS collapse to NO String elements? +/// Rubygems' `authors=` writer keeps only Strings (`grep(String)`), so `[]`, +/// `nil`, `[nil]`, and empty word-arrays (`%w[]`) all yield an empty authors +/// list — the hard `authors may not be empty` error — while `[""]` keeps its +/// String and validates. Fail-open: anything not demonstrably empty passes +/// (a `[42]` would slip through, but `to_ruby` never emits one and bundler +/// still reports it — the heuristic only loses the nicer error). +fn authors_rhs_collapses_empty(rhs: &str) -> bool { + let cleaned = rhs.replace(".freeze", ""); + let cleaned = cleaned.trim(); + let body = cleaned + .strip_prefix("%w") + .or_else(|| cleaned.strip_prefix("%W")) + .unwrap_or(cleaned); + !body + .split(|c: char| c.is_whitespace() || matches!(c, '[' | ']' | '(' | ')' | ',')) + .any(|tok| !tok.is_empty() && tok != "nil") +} + +/// The rubygems-REQUIRED attributes a stub gemspec must assign for bundler to +/// accept it as a path source, returned as the list it is missing (empty = +/// valid). Every bundler major validates path-source gemspecs, so a stub +/// missing these bricks every later `bundle install` — the D4 defect the +/// 2026-08-19 gem live matrix found in ALL served stubs. +/// +/// The bar is EMPIRICAL, verified against rubygems 3.3 / 3.5 / 3.6 +/// (`Gem::Specification#validate`, both packaging modes, in the bundler +/// 1.17 / 2.7 / 4.0 era images): +/// +/// * `summary` — hard `missing value for attribute summary` ONLY when never +/// assigned. The `summary=` writer coerces `nil`/`""` to a present value +/// (empty is at most a warning), so ANY assignment line satisfies it. +/// * `authors` — hard `authors may not be empty` when never assigned (the +/// singular `author =` alias counts) or when every assignment textually +/// collapses to no String elements ([`authors_rhs_collapses_empty`]). +/// +/// A missing `licenses` is only a rubygems WARNING, deliberately not checked +/// here (callers may mention it in advisory text via +/// [`gemspec_assigns_attr`]). Fail-open by construction: only a stub that +/// demonstrably fails the bar is flagged, so a legitimate stub always passes +/// and is written byte-verbatim. +fn gemspec_missing_required_attrs(spec_text: &str) -> Vec<&'static str> { + let mut missing = Vec::new(); + if !gemspec_assigns_attr(spec_text, &["summary"]) { + missing.push("summary"); + } + let author_rhs = gemspec_attr_rhs(spec_text, &["authors", "author"]); + if author_rhs.is_empty() + || author_rhs + .iter() + .all(|rhs| authors_rhs_collapses_empty(rhs)) + { + missing.push("authors"); + } + missing +} + #[cfg(test)] mod tests { use super::*; @@ -2208,7 +2466,10 @@ mod tests { const PRISTINE: &[u8] = b"module Rack\n VERSION = \"3.2.6\"\nend\n"; const PATCHED: &[u8] = b"module Rack\n SOCKET_PATCHED = true\n VERSION = \"3.2.6\"\nend\n"; - const GEMSPEC: &str = "Gem::Specification.new do |s|\n s.name = \"rack\"\n s.version = \"3.2.6\"\n s.summary = \"a modular Ruby web server interface\"\n s.require_paths = [\"lib\"]\nend\n"; + // Every local-stub fixture assigns the rubygems-required `summary` + + // `authors` — as any healthy rubygems-written `specifications/` stub does + // — because the local-build write choke point validates them too. + const GEMSPEC: &str = "Gem::Specification.new do |s|\n s.name = \"rack\"\n s.version = \"3.2.6\"\n s.summary = \"a modular Ruby web server interface\"\n s.authors = [\"Rack maintainers\"]\n s.require_paths = [\"lib\"]\nend\n"; const GEMFILE_DIRECT: &str = "source \"https://rubygems.org\"\n\ngem \"puma\"\ngem \"rack\", \"~> 3.1\"\n"; @@ -2529,6 +2790,89 @@ mod tests { assert!(!root.join(".socket").exists()); } + /// The required-attribute heuristic ([`gemspec_missing_required_attrs`]): + /// flag ONLY what real rubygems hard-fails on (empirically verified + /// against rubygems 3.3/3.5/3.6, see the fn doc) — a stub rubygems + /// tolerates must always pass, whatever its spelling. + #[test] + fn required_attrs_heuristic() { + // The D4 production shape: no summary, no authors. + assert_eq!( + gemspec_missing_required_attrs( + "Gem::Specification.new do |s|\n s.name = \"rack\".freeze\n s.version = \"3.2.6\".freeze\n s.require_paths = [\"lib\".freeze]\nend\n" + ), + vec!["summary", "authors"] + ); + // A real converter/to_ruby stub: `.freeze`-d scalar + array. Valid. + assert_eq!( + gemspec_missing_required_attrs( + "Gem::Specification.new do |s|\n s.summary = \"web server interface\".freeze\n s.authors = [\"A. Person\".freeze, \"B. Person\".freeze]\nend\n" + ), + Vec::<&str>::new() + ); + // Alternate spellings a valid stub may use: another block variable, + // no space around `=`, the singular `author =` alias, %w arrays. + assert_eq!( + gemspec_missing_required_attrs( + "Gem::Specification.new do |spec|\n spec.summary=\"x\"\n spec.author = \"A. Person\"\nend\n" + ), + Vec::<&str>::new() + ); + assert_eq!( + gemspec_missing_required_attrs("s.summary = \"x\"\ns.authors = %w[alice bob]\n"), + Vec::<&str>::new() + ); + // A `#` inside a string literal is CONTENT, not a comment — this + // valid stub must never be judged missing (scanning raw lines, + // anchored to the line start, instead of comment-stripping). + assert_eq!( + gemspec_missing_required_attrs( + "s.summary = \"#1 Ruby web server\".freeze\ns.authors = [\"D. #2 Person\".freeze]\n" + ), + Vec::<&str>::new() + ); + // One present, one absent → only the absent one is named. + assert_eq!( + gemspec_missing_required_attrs("s.summary = \"x\".freeze\n"), + vec!["authors"] + ); + // Rubygems TOLERATES nil/empty summary (the writer coerces; empty is + // a warning) and an empty-STRING author ([""] keeps its String), so + // none of these flag. + assert_eq!( + gemspec_missing_required_attrs("s.summary = \"\".freeze\ns.authors = [\"\"]\n"), + Vec::<&str>::new() + ); + assert_eq!( + gemspec_missing_required_attrs("s.summary = nil\ns.authors = [\"a\"]\n"), + Vec::<&str>::new() + ); + // Rubygems HARD-FAILS an authors list with no String elements + // (`authors may not be empty`): [], nil, [nil], %w[] all flag. + for empty_authors in ["[]", "[].freeze", "nil", "[nil]", "%w[]", "%W()"] { + assert_eq!( + gemspec_missing_required_attrs(&format!( + "s.summary = \"x\"\ns.authors = {empty_authors}\n" + )), + vec!["authors"], + "authors = {empty_authors} must flag" + ); + } + // Commented-out assignments, `==` comparisons, and mid-line mentions + // are not assignments. + assert_eq!( + gemspec_missing_required_attrs( + "# s.summary = \"x\"\nraise if s.authors == [\"x\"]\nfoo(s.summary = \"x\")\n" + ), + vec!["summary", "authors"] + ); + // Longer identifiers are not the attribute (`.authors` != `.author`). + assert_eq!( + gemspec_missing_required_attrs("s.summary_text = \"x\"\ns.author_email = \"x\"\n"), + vec!["summary", "authors"] + ); + } + #[tokio::test] async fn test_refuses_native_extensions() { let (_tmp, root, installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; @@ -2988,7 +3332,7 @@ mod tests { const PRISTINE_318: &[u8] = b"module Rack\n VERSION = \"3.1.8\"\nend\n"; const PATCHED_318: &[u8] = b"module Rack\n SOCKET_PATCHED = true\n VERSION = \"3.1.8\"\nend\n"; - const GEMSPEC_318: &str = "Gem::Specification.new do |s|\n s.name = \"rack\"\n s.version = \"3.1.8\"\n s.require_paths = [\"lib\"]\nend\n"; + const GEMSPEC_318: &str = "Gem::Specification.new do |s|\n s.name = \"rack\"\n s.version = \"3.1.8\"\n s.summary = \"a modular Ruby web server interface\"\n s.authors = [\"Rack maintainers\"]\n s.require_paths = [\"lib\"]\nend\n"; // Embedded VERBATIM from the spike pair // `spikes/gem-checksums/path-with-checksums/{before,after}/` (bundler @@ -3521,7 +3865,7 @@ mod tests { const PRISTINE_PUMA: &[u8] = b"module Puma\n VERSION = \"6.4.2\"\nend\n"; const PATCHED_PUMA: &[u8] = b"module Puma\n SOCKET_PATCHED = true\n VERSION = \"6.4.2\"\nend\n"; - const GEMSPEC_PUMA: &str = "Gem::Specification.new do |s|\n s.name = \"puma\"\n s.version = \"6.4.2\"\n s.require_paths = [\"lib\"]\nend\n"; + const GEMSPEC_PUMA: &str = "Gem::Specification.new do |s|\n s.name = \"puma\"\n s.version = \"6.4.2\"\n s.summary = \"a fast, concurrent web server\"\n s.authors = [\"Puma maintainers\"]\n s.require_paths = [\"lib\"]\nend\n"; fn puma_rel() -> String { format!(".socket/vendor/gem/{UUID_PUMA}/puma-6.4.2") @@ -4002,10 +4346,17 @@ mod tests { use crate::api::client::{ApiClient, ApiClientOptions}; use crate::vendor::VendorSource; - /// A valid path-source stub (no native extensions). - const SERVICE_STUB: &[u8] = b"# -*- encoding: utf-8 -*-\n# stub: rack 3.2.6 ruby lib\n\nGem::Specification.new do |s|\n s.name = \"rack\".freeze\n s.version = \"3.2.6\".freeze\n s.require_paths = [\"lib\".freeze]\nend\n"; + /// A valid path-source stub (no native extensions; assigns the + /// rubygems-required `summary` + `authors`, which every bundler major + /// validates on path-source gemspecs). + const SERVICE_STUB: &[u8] = b"# -*- encoding: utf-8 -*-\n# stub: rack 3.2.6 ruby lib\n\nGem::Specification.new do |s|\n s.name = \"rack\".freeze\n s.version = \"3.2.6\".freeze\n s.summary = \"a modular Ruby web server interface\".freeze\n s.authors = [\"Rack maintainers\".freeze]\n s.licenses = [\"MIT\".freeze]\n s.require_paths = [\"lib\".freeze]\nend\n"; /// A stub that declares native extensions (must be refused). const SERVICE_STUB_NATIVE: &[u8] = b"Gem::Specification.new do |s|\n s.name = \"rack\".freeze\n s.version = \"3.2.6\".freeze\n s.extensions = [\"ext/rack/extconf.rb\"]\nend\n"; + /// The DEFECTIVE stub shape production served as of 2026-08-19 (gem + /// live-matrix defect D4): it never assigns the rubygems-required + /// `summary` / `authors` (nor `licenses`), so bundler's path-source + /// validation rejects it and every post-vendor `bundle install` exits 1. + const SERVICE_STUB_INVALID: &[u8] = b"# -*- encoding: utf-8 -*-\n# stub: rack 3.2.6 ruby lib\n\nGem::Specification.new do |s|\n s.name = \"rack\".freeze\n s.version = \"3.2.6\".freeze\n s.require_paths = [\"lib\".freeze]\nend\n"; fn sri_sha512(bytes: &[u8]) -> String { use base64::Engine as _; @@ -4146,8 +4497,10 @@ mod tests { } /// Service success: the prebuilt `.gem` is extracted into the copy dir, the - /// served stub is written as `rack.gemspec`, the Gemfile + lock are wired, - /// and a `vendor_prebuilt_downloaded` advisory is emitted — WITHOUT a local + /// served stub is written as `rack.gemspec` BYTE-VERBATIM (a valid stub — + /// one assigning `summary`/`authors` — must pass the required-attribute + /// validation untouched), the Gemfile + lock are wired, and a + /// `vendor_prebuilt_downloaded` advisory is emitted — WITHOUT a local /// install (a deliberately-missing `installed_dir`). #[tokio::test] async fn service_success_extracts_gem_and_wires_lock() { @@ -4333,6 +4686,320 @@ mod tests { ); } + /// D4 (gem live-matrix 2026-08-19): explicit `service` mode + a served stub + /// that never assigns the rubygems-required `summary`/`authors` refuses + /// with its own `vendor_prebuilt_stub_invalid` code, naming the missing + /// attributes — writing it verbatim would make every later `bundle install` + /// exit 1 (all bundler majors validate path-source gemspecs). No partial + /// artifacts are left and the lock is untouched. + #[tokio::test] + async fn service_stub_invalid_service_mode_hard_fails() { + let (_tmp, root, installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; + let gem = make_gem(&[("lib/rack.rb", PATCHED)]); + let sri = sri_sha512(&gem); + let stub_sri = sri_sha512(SERVICE_STUB_INVALID); + let server = wiremock::MockServer::start().await; + mount_gem_granted(&server, &gem, &sri, Some((SERVICE_STUB_INVALID, &stub_sri))).await; + let sources = PatchSources::blobs_only(&blobs); + + let outcome = vendor_gem( + PURL, + &installed, + &root, + &record, + &sources, + "2026-06-09T00:00:00Z", + false, + false, + Some(&gem_service_cfg( + &server.uri(), + VendorSource::Service, + false, + )), + ) + .await; + let (code, detail) = unwrap_refused(outcome); + assert_eq!(code, "vendor_prebuilt_stub_invalid"); + assert!( + detail.contains("summary") && detail.contains("authors"), + "the refusal must name the missing attributes: {detail}" + ); + assert!(!root.join(format!(".socket/vendor/gem/{UUID}")).exists()); + // The lock is untouched. + assert_eq!( + tokio::fs::read_to_string(root.join(GEMFILE_LOCK)) + .await + .unwrap(), + LOCK_DIRECT + ); + } + + /// D4 under the default `auto`: an INVALID served stub is treated exactly + /// like a MISSING one — fall back to the LOCAL build (installed gem + + /// locally derived stub) — but with a LOUD `vendor_prebuilt_stub_invalid` + /// warning naming the served-stub defect. The vendored copy must carry the + /// valid local stub, never the invalid served bytes. + #[tokio::test] + async fn service_stub_invalid_auto_falls_back_to_build() { + let (_tmp, root, installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; + let gem = make_gem(&[("lib/rack.rb", PATCHED)]); + let sri = sri_sha512(&gem); + let stub_sri = sri_sha512(SERVICE_STUB_INVALID); + let server = wiremock::MockServer::start().await; + mount_gem_granted(&server, &gem, &sri, Some((SERVICE_STUB_INVALID, &stub_sri))).await; + let sources = PatchSources::blobs_only(&blobs); + + let outcome = vendor_gem( + PURL, + &installed, + &root, + &record, + &sources, + "2026-06-09T00:00:00Z", + false, + false, + Some(&gem_service_cfg(&server.uri(), VendorSource::Auto, false)), + ) + .await; + let (result, entry, warnings) = unwrap_done(outcome); + assert!(result.success, "auto must fall back: {:?}", result.error); + assert!(entry.is_some()); + assert_eq!(tokio::fs::read(copy_lib(&root)).await.unwrap(), PATCHED); + assert_eq!( + tokio::fs::read_to_string(copy_gemspec(&root)) + .await + .unwrap(), + GEMSPEC, + "the vendored gemspec must be the LOCAL stub, not the invalid served bytes" + ); + let warning = warnings + .iter() + .find(|w| w.code == "vendor_prebuilt_stub_invalid") + .expect("auto fallback must warn loudly about the invalid served stub"); + assert!( + warning.detail.contains("summary") && warning.detail.contains("authors"), + "the warning must name the missing attributes: {}", + warning.detail + ); + } + + /// D4 + `auto` + the gem NOT installed (a `missing_install` staging-style + /// dir): the local-build fallback has no stub to derive, and the refusal + /// must be TRUTHFUL — it carries the served-stub defect (a `Refused` + /// outcome has no warnings channel, so the detail is the diagnostic's + /// only route into the envelope) and the install-the-gem remedy, never + /// `gem_spec_missing`'s circular "use --vendor-source=service" advice + /// (service refuses on the same defect). + #[tokio::test] + async fn service_stub_invalid_auto_not_installed_refuses_truthfully() { + let (_tmp, root, _installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; + let gem = make_gem(&[("lib/rack.rb", PATCHED)]); + let sri = sri_sha512(&gem); + let stub_sri = sri_sha512(SERVICE_STUB_INVALID); + let server = wiremock::MockServer::start().await; + mount_gem_granted(&server, &gem, &sri, Some((SERVICE_STUB_INVALID, &stub_sri))).await; + let sources = PatchSources::blobs_only(&blobs); + + let outcome = vendor_gem( + PURL, + &missing_install(&root), + &root, + &record, + &sources, + "2026-06-09T00:00:00Z", + false, + false, + Some(&gem_service_cfg(&server.uri(), VendorSource::Auto, false)), + ) + .await; + let (code, detail) = unwrap_refused(outcome); + assert_eq!(code, "vendor_prebuilt_stub_invalid"); + assert!( + detail.contains("summary") && detail.contains("authors"), + "the refusal must carry the served-stub defect: {detail}" + ); + assert!( + detail.contains("not installed locally") && detail.contains("install the gem"), + "the refusal must advise installing the gem: {detail}" + ); + assert!( + !detail.contains("--vendor-source=service"), + "circular advice (service refuses on the same defect): {detail}" + ); + assert!(!root.join(".socket").exists()); + } + + /// D4 + explicit `service` + the gem NOT installed: the hard refusal is + /// the same as the installed case — installation is irrelevant to + /// `service` mode, which never falls back. + #[tokio::test] + async fn service_stub_invalid_service_mode_not_installed_hard_fails() { + let (_tmp, root, _installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; + let gem = make_gem(&[("lib/rack.rb", PATCHED)]); + let sri = sri_sha512(&gem); + let stub_sri = sri_sha512(SERVICE_STUB_INVALID); + let server = wiremock::MockServer::start().await; + mount_gem_granted(&server, &gem, &sri, Some((SERVICE_STUB_INVALID, &stub_sri))).await; + let sources = PatchSources::blobs_only(&blobs); + + let outcome = vendor_gem( + PURL, + &missing_install(&root), + &root, + &record, + &sources, + "2026-06-09T00:00:00Z", + false, + false, + Some(&gem_service_cfg( + &server.uri(), + VendorSource::Service, + false, + )), + ) + .await; + let (code, detail) = unwrap_refused(outcome); + assert_eq!(code, "vendor_prebuilt_stub_invalid"); + assert!( + detail.contains("--vendor-source=auto"), + "the service refusal names the auto/build remedy: {detail}" + ); + assert!(!root.join(".socket").exists()); + } + + /// SECURITY: the local stub gemspec is derived from `installed_dir` ONLY + /// when it sits inside a real gem home's `gems/` dir. For an auto-fetch + /// staging dir (`/-`), walking two + /// parents up would escape into the SHARED temp root, where + /// `specifications/.gemspec` is a predictable, attacker-plantable + /// path whose contents would be committed into the project and eval'd as + /// Ruby by every later `bundle install`. The planted spec must never be + /// consumed: with no service configured the vendor refuses + /// `gem_spec_missing`. + #[tokio::test] + async fn planted_spec_outside_gem_home_is_not_consumed() { + let (tmp, root, _installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; + let base = tmp.path(); + // The auto-fetch staging shape: /stage/-, with + // the pristine bytes present (the parent is NOT named `gems`). + let staged = base.join("stage/rack-3.2.6"); + tokio::fs::create_dir_all(staged.join("lib")).await.unwrap(); + tokio::fs::write(staged.join("lib/rack.rb"), PRISTINE) + .await + .unwrap(); + // The attacker's plant, at exactly where an unguarded + // parent-of-parent derivation would look: a VALID stub, so consuming + // it would "succeed". + tokio::fs::create_dir_all(base.join("specifications")) + .await + .unwrap(); + tokio::fs::write(base.join("specifications/rack-3.2.6.gemspec"), GEMSPEC) + .await + .unwrap(); + + let (code, detail) = + unwrap_refused(run_vendor_purl(PURL, &root, &blobs, &staged, &record, false).await); + assert_eq!( + code, "gem_spec_missing", + "the planted spec outside a gem home must not be consumed: {detail}" + ); + assert!(!root.join(".socket").exists()); + } + + /// The write choke point validates the LOCALLY-derived stub too: a + /// corrupted `specifications/` stub missing the required attributes is an + /// honest `gem_spec_invalid` refusal naming the file — never a vendored + /// copy bundler will reject at install time. + #[tokio::test] + async fn local_stub_invalid_refuses_with_gem_spec_invalid() { + let (_tmp, root, installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; + let spec = installed + .parent() + .unwrap() + .parent() + .unwrap() + .join("specifications/rack-3.2.6.gemspec"); + tokio::fs::write( + &spec, + "Gem::Specification.new do |s|\n s.name = \"rack\"\n s.version = \"3.2.6\"\n s.require_paths = [\"lib\"]\nend\n", + ) + .await + .unwrap(); + + let (code, detail) = + unwrap_refused(run_vendor(&root, &blobs, &installed, &record, false).await); + assert_eq!(code, "gem_spec_invalid"); + assert!( + detail.contains("summary") + && detail.contains("authors") + && detail.contains("rack-3.2.6.gemspec"), + "the refusal names the file and the missing attributes: {detail}" + ); + assert!(!root.join(".socket").exists()); + assert_eq!( + tokio::fs::read_to_string(root.join(GEMFILE)).await.unwrap(), + GEMFILE_DIRECT + ); + assert_eq!( + tokio::fs::read_to_string(root.join(GEMFILE_LOCK)) + .await + .unwrap(), + LOCK_DIRECT + ); + } + + /// D4 heal: a project vendored PRE-hardening carries the invalid served + /// stub on disk. The idempotent hot path must not re-bless it as + /// `already_vendored`: the on-disk stub fails the required-attribute bar, + /// routing into the artifact-only rebuild, which rewrites a valid stub + /// with the pair edit and the ledger entry untouched. + #[tokio::test] + async fn wired_copy_with_invalid_stub_is_rebuilt() { + let (_tmp, root, installed, blobs, record) = fixture(GEMFILE_DIRECT, LOCK_DIRECT).await; + let (result, entry, _) = + unwrap_done(run_vendor(&root, &blobs, &installed, &record, false).await); + assert!(result.success, "{:?}", result.error); + assert!(entry.is_some()); + let gemfile_wired = tokio::fs::read(root.join(GEMFILE)).await.unwrap(); + let lock_wired = tokio::fs::read(root.join(GEMFILE_LOCK)).await.unwrap(); + + // Simulate the pre-fix victim: the served invalid stub on disk. + tokio::fs::write(copy_gemspec(&root), SERVICE_STUB_INVALID) + .await + .unwrap(); + + let (result2, entry2, warnings2) = + unwrap_done(run_vendor(&root, &blobs, &installed, &record, false).await); + assert!(result2.success, "{:?}", result2.error); + assert!( + entry2.is_none(), + "artifact-only rebuild must not re-record a ledger entry" + ); + assert!( + warnings2 + .iter() + .any(|w| w.code == "vendor_artifact_rebuilt"), + "the heal must surface as a rebuild, not a silent no-op: {warnings2:?}" + ); + assert_eq!( + tokio::fs::read_to_string(copy_gemspec(&root)) + .await + .unwrap(), + GEMSPEC, + "the invalid on-disk stub must be replaced with the valid local stub" + ); + assert_eq!( + tokio::fs::read(root.join(GEMFILE)).await.unwrap(), + gemfile_wired, + "the heal must not touch the Gemfile" + ); + assert_eq!( + tokio::fs::read(root.join(GEMFILE_LOCK)).await.unwrap(), + lock_wired, + "the heal must not touch Gemfile.lock" + ); + } + /// `auto` + a not-built service status falls back to the local build. #[tokio::test] async fn service_unavailable_auto_falls_back_to_build() { diff --git a/docs/testing/vendored-production-e2e.md b/docs/testing/vendored-production-e2e.md index 5667713b..695152e6 100644 --- a/docs/testing/vendored-production-e2e.md +++ b/docs/testing/vendored-production-e2e.md @@ -68,7 +68,7 @@ hosted suite). | pip (requirements.txt) | urllib3@1.26.18 | `pip install --no-index -r requirements.txt` | ✅ full | | uv (uv.lock) | urllib3@1.26.18 | `uv sync --frozen --offline` | ✅ full | | cargo (`[patch.crates-io]`) | traitobject@0.1.1 | `cargo fetch --offline --locked` (see note) | ✅ full | -| bundler | activestorage@6.0.3 | — (deferred: production's served gem-stub gemspec is invalid, see below) | ⚠️ vendor succeeds; delivery proof deferred to the stub-hardening fix PR | +| bundler | activestorage@6.0.3 | frozen `bundle install`, fresh empty `BUNDLE_PATH` | ✅ full | | go | — | — | zero-patch assertion (no free golang patches) | | deno | — | — | negative assertion (unsupported) | | maven / nuget / composer | — | — | canary (no free production patches) | @@ -84,7 +84,8 @@ directory. ## Known issues this suite surfaced All were found against real production + real toolchains; none is a test bug. -The first two are fixed; the third is open with a stacked fix PR pending. +The first two are fixed; the third is mitigated CLI-side (the served artifact +is still defective server-side). ### 1. `pnpm` >= 11 — vendored `overrides` land in the wrong file (CLI) — FIXED @@ -118,21 +119,36 @@ vendors like a bare purl. The suite's original pin (`activestorage@7.0.2.2` / `2535d43d-67ce-4944-be27-c19e113997fb`) was withdrawn on 2026-08-14; the 2026-08-18 catalog republish REPLACED it, and the suite was re-pinned to `activestorage@6.0.3` / -`15e960b5-f432-4b6c-b8aa-534a2b419323`. The vendor now succeeds live and the -leg's failure-tolerance branch is vestigial. The leg's upgrade to a full fresh-dir -`bundle install` delivery proof is deferred to the stacked stub-hardening fix -PR because of issue 3 below; until then the leg keeps its tolerant shape and -its `SOCKET_PATCH_VENDORED_E2E_GEM_STRICT` knob. +`15e960b5-f432-4b6c-b8aa-534a2b419323`. The vendor succeeds live and the leg +was upgraded to the full fresh-dir `bundle install` delivery proof +(`gem_bundler_vendored_install_proof`), retiring its failure-tolerance branch +and its `SOCKET_PATCH_VENDORED_E2E_GEM_STRICT` knob. -### 3. `gem` — the served gem-stub gemspec is invalid (SERVER + CLI hardening) — OPEN +### 3. `gem` — the served gem-stub gemspec is invalid (SERVER; CLI mitigated) Discovered 2026-08-19 while upgrading the gem leg to a full delivery proof: the gem-stub-gemspec artifact production serves is invalid — it is missing -`summary`/`authors`, which rubygems validation requires — so bundler rejects -the vendored `path:` source and `bundle install` exits 1 on every bundler -major. The fix (server-side stub correction plus CLI-side hardening) lands in -a separate stacked PR; that PR carries the full -fresh-dir-frozen-`bundle install` install proof as its regression test. +`summary`/`authors`, which rubygems validation requires — so writing it +verbatim makes bundler reject the vendored `path:` source and +`bundle install` exit 1 on every bundler major. + +**Mitigated CLI-side**: the gem vendor backend now validates BOTH stub sources +at the write choke point (a conservative textual check matched to what +rubygems 3.3–3.6 actually hard-fails — empirically verified in the bundler-era +docker images). `--vendor-source auto` detects the served defect, warns +(`vendor_prebuilt_stub_invalid`), and falls back to the local build — which is +how `gem_bundler_vendored_install_proof` passes against production today — +while explicit `--vendor-source service` refuses with +`vendor_prebuilt_stub_invalid`. When the gem is also not installed (no local +stub to derive), `auto` refuses with the same code, the served defect named, +and the install-the-gem remedy; a corrupted LOCAL `specifications/` stub +refuses `gem_spec_invalid`. Re-vendoring a project that committed a defective +stub pre-hardening re-validates the ON-DISK stub and rebuilds the artifact +with a valid one. The leg's route-attribution assertion (exactly one of +`vendor_prebuilt_downloaded` / `vendor_prebuilt_stub_invalid`) auto-retires +the fallback expectation once the depscan stub-generator fix deploys and the +rebuilt artifacts serve valid stubs — the same leg then exercises the service +artifact directly. ## Running @@ -154,7 +170,6 @@ installs from contending on the shared cache sandbox. | Variable | Effect | |----------|--------| | `SOCKET_PATCH_VENDORED_E2E_STRICT=1` | Turn every "toolchain missing" soft-skip into a hard failure. | -| `SOCKET_PATCH_VENDORED_E2E_GEM_STRICT=1` | Promote any gem vendor failure to a hard failure (the leg's tolerance branch is vestigial since #172; the knob stays until the stub-hardening PR lands the full install proof). | | `SOCKET_PATCH_VENDORED_E2E_CANARY_STRICT=1` | Fail when maven/nuget/composer gain their first free published patch. | The suite forces `SOCKET_NO_CONFIG=true` and scrubs every ambient `SOCKET_*`