feat(gem): hosted CHECKSUMS locks converge — patch-registry GEM section + dependency pin, frozen-installable - #212
Merged
Mikola Lysenko (mikolalysenko) merged 8 commits intoAug 19, 2026
Conversation
…xit-37 mixed state Red half of KL1 (bundler-4 DEFAULT lock => mainstream hosted-gem path): - 4 new core unit tests pinning the fully converged rewrite on a CHECKSUMS lock: patch-registry GEM section holding the moved spec (+sublines), '<name> (= <ver>)!' DEPENDENCIES pin (rewritten, or added sorted for a transitive dep), patched CHECKSUMS sha, no redirect_gem_frozen_install caveat, converged re-run a no-op, and rotated-grant refresh of the converged lock's GEM remote (redirect_gemfile_lock_source_url). - e2e canary FLIPPED per its own header: was gem_hosted_checksums_lock_pins_patched_sha_but_bundler_refuses_mixed_state (pinning exit 37), now gem_hosted_checksums_lock_converges_and_installs_frozen_and_unfrozen — keeps the ledger-original rewrite-half asserts and now demands the converged lock plus green FROZEN (BUNDLE_FROZEN=true, lock byte-identical — the exit-16 two-step gone) and UNFROZEN fresh installs of the patched bytes. All five captured red at this commit (unit: mixed-state lock output; e2e: converged-GEM-section assert against the real 4.0.15 lock). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…on + dependency pin, frozen-installable Implements the verified fix shape for KL1 (exit 37 'mismatched checksums' on the bundler-4 DEFAULT lock, exit 16 two-step under frozen/deployment): when a CHECKSUMS pin lands (or is already at target), converge_gem_lock_source rewrites the lock into what bundler itself writes after an install from the redirected Gemfile — - the dep's spec entry (+ dependency sublines) moves out of the upstream GEM section into a patch-registry GEM section (remote: <index-url>), ledger edit redirect_gemfile_lock_gem_source with the upstream remote as revert original; - DEPENDENCIES pins '<name> (= <version>)!' (rewritten, or added in bundler's sorted position for a transitive dep), ledger edit redirect_gemfile_lock_dependency_pin; - rotation-aware and idempotent: a section whose remote matches the token-wildcard pattern is recognized as ours (never duplicated) and refreshed in place under a rotated grant (redirect_gemfile_lock_source_url), CRLF preserved throughout; - fail-soft: an unattributable spec (absent, duplicated, legacy multi-remote section, no DEPENDENCIES) leaves today's mixed state. redirect_gem_frozen_install now fires only on a genuinely MIXED pair (pre-CHECKSUMS locks) — a converged pair is frozen-installable as written, so the caveat is dropped there. Flipped e2e canary (real host bundler 4.0.15) proves the converged pair fresh-installs patched bytes both FROZEN (BUNDLE_FROZEN=true, lock byte-identical) and unfrozen. NOTE: the depscan TS twin (registry-rewrite gem.ts) must be ported to match — cross-repo follow-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Reviewer P2 (second half): the convergence work introduced 3 more rustfmt diffs on its own lines. Scoped rustfmt run on the touched file only; no behavior change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… fail soft, not converge on stale indices Reviewer nit: converge_gem_lock_source runs the DEPENDENCIES pin first on the premise those lines sit after the GEM sections (so the later spec-move indices never shift). Bundler always writes sources first, but a hand-edited lock with DEPENDENCIES before GEM breaks the premise — the transitive-dep pin INSERT would shift the parsed spec/remote/end indices before the spec move reads them. Pin the fail-soft contract: checksum pinned, GEM attribution untouched, frozen-install caveat, no convergence edits. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… GEM section One-line ordering guard (reviewer nit): converge_gem_lock_source edits DEPENDENCIES first because bundler writes source sections before it — a hand-edited lock violating that order would leave the spec-move splicing on indices the pin insert had already shifted. Guard: deps_start before the spec section's end routes to the existing fail-soft mixed path (checksum pin + frozen-install caveat, lock shape untouched). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.
Bugbot Autofix is ON. A cloud agent has been kicked off to fix the reported issue.
Comment @cursor review or bugbot run to trigger another review on this PR
Reviewed by Cursor Bugbot for commit f5ddcb6. Configure here.
The converged-lock rewrite changes gem/bundler/basic's expected output: the lock now carries a patch-registry GEM section, the `rails (= 7.0.0)!` DEPENDENCIES pin, and two new ledger edit kinds (redirect_gemfile_lock_dependency_pin, redirect_gemfile_lock_gem_source) with faithful originals. Caught by the workspace coverage job; the shape matches what the bundler-matrix campaign verified frozen-installs clean on bundler 4.0.18. Cross-repo: depscan's TS gem.ts twin must land the same convergence and re-bless its copy of this fixture in lockstep (shared golden contract). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Collaborator
Author
|
CI fix pushed (93e1203): the workspace coverage job caught Reminder from the PR body: depscan's TS |
Bugbot round: converge_gem_lock_source wrote ov.index_url verbatim into lock remote: lines (and the Gemfile source string always did the same) with no grammar check. New is_valid_gem_index_url — http(s) scheme, no quote/backslash/whitespace/control chars — gates the gem arm at intake, twin of is_valid_cargo_index_url; malformed URLs skip the dependency with redirect_gem_invalid_index_url. Unit test covers quote, backslash, newline-injection, space, and non-http schemes. Cross-repo: the depscan gem.ts twin needs the same intake gate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Mikola Lysenko (mikolalysenko)
merged commit Aug 19, 2026
58cd923
into
fix/gem-hosted-grant-rotation
41 checks passed
Mikola Lysenko (mikolalysenko)
deleted the
feat/gem-hosted-converged-lock
branch
August 19, 2026 14:54
Mikola Lysenko (mikolalysenko)
added a commit
that referenced
this pull request
Aug 19, 2026
… no more nested Gemfile sources (#211) * test(gem): red — hosted re-scan under a rotated grant nests source blocks with the CLI's empty token Regression tests for the P0: the CLI's only production DepOverride construction site (scan/hosted.rs) leaves token empty, so the gem rewriter's grant-rotation idempotency guard (which wildcards only non-empty rotating segments) never recognizes the previous grant's source block. A re-scan under a rotated grant wraps the old block's indented gem line in a NEW nested source block, keeps the stale token URL live, corrupts the ledger revert chain, and reports success. - core unit: rotated re-run with CLI-shaped (empty-token) overrides must refresh the URL in place — currently nests (2 blocks). - core unit: gems.rb/Gemfile identical twins + rotated grant + empty token must not be trapped behind spellings-diverge — currently is. - e2e (real host bundler): scan A -> re-scan A (byte-idempotent, holds today) -> re-scan B rotated — currently nests exactly like the campaign repro (hosted-b1/b2/b4 idempotent-rerun), then fresh install of the rotated pair. All three captured red at this commit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(gem): hosted grant-rotation guard no longer depends on the CLI populating DepOverride.token Defense in depth for the P0 rotated-grant nesting corruption: - core: gem_index_url_pattern now derives the grant-token path level from the index URL itself (the segment immediately preceding the patch uuid — production's /patch-registry/gem/{token}/{uuid}/ shape) and wildcards it unconditionally, so the block recognizer, the refresh-in-place branch, and gem_spelling_residue's footprint erasure all survive a caller that leaves token empty. - cli: scan/hosted.rs (the only production DepOverride construction site) now recovers the token via the new pub grant_token_path_segment helper from the registryOverride indexUrl (artifact URL fallback) instead of hard-coding String::new() — every path-token'd ecosystem gets a truthful token. - polish: the redirect_gem_source_option refusal prescribes 'socket-patch vendor --revert' when the blocking path: option is socket-patch's own .socket/vendor wiring. Red tests from the previous commit now pass; the rotated-grant e2e leg (token A -> A -> B, same uuid, real host bundler) proves exactly one source block, a redirect_gemfile_source_url ledger edit, and a green fresh-checkout install of the patched bytes. NOTE: the depscan TS twin (registry-rewrite gem.ts) must be ported to match — cross-repo follow-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style(gem): rustfmt the rotation-guard code and gem e2e helpers Reviewer P2: the new code introduced 5 rustfmt diffs on its own lines (main's versions of both files are fmt-clean). Scoped rustfmt run on the two touched files only; no behavior change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(gem): source-option refusal states vendor --revert's whole-project blast radius; expect() in new capstone reads Bugbot round: (1) the socket-vendored-wiring refusal now prescribes the per-gem eject (remove <purl>) first and says vendor --revert reverts EVERY vendored dependency, not just the blocking gem; unit test pins both statements. (2) the rotation capstone's new file reads carry step-labeled expect()s so a CI failure names the step. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(gem): hosted CHECKSUMS locks converge — patch-registry GEM section + dependency pin, frozen-installable (#212) * test(gem): red — hosted CHECKSUMS lock must converge, not leave the exit-37 mixed state Red half of KL1 (bundler-4 DEFAULT lock => mainstream hosted-gem path): - 4 new core unit tests pinning the fully converged rewrite on a CHECKSUMS lock: patch-registry GEM section holding the moved spec (+sublines), '<name> (= <ver>)!' DEPENDENCIES pin (rewritten, or added sorted for a transitive dep), patched CHECKSUMS sha, no redirect_gem_frozen_install caveat, converged re-run a no-op, and rotated-grant refresh of the converged lock's GEM remote (redirect_gemfile_lock_source_url). - e2e canary FLIPPED per its own header: was gem_hosted_checksums_lock_pins_patched_sha_but_bundler_refuses_mixed_state (pinning exit 37), now gem_hosted_checksums_lock_converges_and_installs_frozen_and_unfrozen — keeps the ledger-original rewrite-half asserts and now demands the converged lock plus green FROZEN (BUNDLE_FROZEN=true, lock byte-identical — the exit-16 two-step gone) and UNFROZEN fresh installs of the patched bytes. All five captured red at this commit (unit: mixed-state lock output; e2e: converged-GEM-section assert against the real 4.0.15 lock). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(gem): hosted CHECKSUMS locks converge — patch-registry GEM section + dependency pin, frozen-installable Implements the verified fix shape for KL1 (exit 37 'mismatched checksums' on the bundler-4 DEFAULT lock, exit 16 two-step under frozen/deployment): when a CHECKSUMS pin lands (or is already at target), converge_gem_lock_source rewrites the lock into what bundler itself writes after an install from the redirected Gemfile — - the dep's spec entry (+ dependency sublines) moves out of the upstream GEM section into a patch-registry GEM section (remote: <index-url>), ledger edit redirect_gemfile_lock_gem_source with the upstream remote as revert original; - DEPENDENCIES pins '<name> (= <version>)!' (rewritten, or added in bundler's sorted position for a transitive dep), ledger edit redirect_gemfile_lock_dependency_pin; - rotation-aware and idempotent: a section whose remote matches the token-wildcard pattern is recognized as ours (never duplicated) and refreshed in place under a rotated grant (redirect_gemfile_lock_source_url), CRLF preserved throughout; - fail-soft: an unattributable spec (absent, duplicated, legacy multi-remote section, no DEPENDENCIES) leaves today's mixed state. redirect_gem_frozen_install now fires only on a genuinely MIXED pair (pre-CHECKSUMS locks) — a converged pair is frozen-installable as written, so the caveat is dropped there. Flipped e2e canary (real host bundler 4.0.15) proves the converged pair fresh-installs patched bytes both FROZEN (BUNDLE_FROZEN=true, lock byte-identical) and unfrozen. NOTE: the depscan TS twin (registry-rewrite gem.ts) must be ported to match — cross-repo follow-up. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * style(gem): rustfmt the lock-convergence code and its units Reviewer P2 (second half): the convergence work introduced 3 more rustfmt diffs on its own lines. Scoped rustfmt run on the touched file only; no behavior change. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(gem): red — a hand-edited lock with DEPENDENCIES before GEM must fail soft, not converge on stale indices Reviewer nit: converge_gem_lock_source runs the DEPENDENCIES pin first on the premise those lines sit after the GEM sections (so the later spec-move indices never shift). Bundler always writes sources first, but a hand-edited lock with DEPENDENCIES before GEM breaks the premise — the transitive-dep pin INSERT would shift the parsed spec/remote/end indices before the spec move reads them. Pin the fail-soft contract: checksum pinned, GEM attribution untouched, frozen-install caveat, no convergence edits. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(gem): lock convergence bails when DEPENDENCIES precedes the dep's GEM section One-line ordering guard (reviewer nit): converge_gem_lock_source edits DEPENDENCIES first because bundler writes source sections before it — a hand-edited lock violating that order would leave the spec-move splicing on indices the pin insert had already shifted. Guard: deps_start before the spec section's end routes to the existing fail-soft mixed path (checksum pin + frozen-install caveat, lock shape untouched). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test(gem): bless the shared golden fixture to the converged lock shape The converged-lock rewrite changes gem/bundler/basic's expected output: the lock now carries a patch-registry GEM section, the `rails (= 7.0.0)!` DEPENDENCIES pin, and two new ledger edit kinds (redirect_gemfile_lock_dependency_pin, redirect_gemfile_lock_gem_source) with faithful originals. Caught by the workspace coverage job; the shape matches what the bundler-matrix campaign verified frozen-installs clean on bundler 4.0.18. Cross-repo: depscan's TS gem.ts twin must land the same convergence and re-bless its copy of this fixture in lockstep (shared golden contract). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(gem): gate service-supplied index URLs before any Gemfile/lock write Bugbot round: converge_gem_lock_source wrote ov.index_url verbatim into lock remote: lines (and the Gemfile source string always did the same) with no grammar check. New is_valid_gem_index_url — http(s) scheme, no quote/backslash/whitespace/control chars — gates the gem arm at intake, twin of is_valid_cargo_index_url; malformed URLs skip the dependency with redirect_gem_invalid_index_url. Unit test covers quote, backslash, newline-injection, space, and non-http schemes. Cross-repo: the depscan gem.ts twin needs the same intake gate. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #211 (
fix/gem-hosted-grant-rotation) — merge that first, then retarget/merge this.Defect
On CHECKSUMS-era locks (bundler >= 2.6 writes the section; bundler 4 by default) the hosted gem redirect left the Gemfile/lock pair in a MIXED state: CHECKSUMS pinned the patched sha while the lock's GEM section still attributed the gem to the upstream remote. Bundler refuses that pair outright — the prescribed unfrozen
bundle installexits 37 "mismatched checksums", and a frozen/deployment install exits 16 — so the documented remediation path was broken exactly where modern bundler lives.Root cause
rewrite_gem(crates/socket-patch-core/src/patch/redirect/mod.rs) only rewrote the Gemfile source block and the CHECKSUMS sha line; nothing moved the dep's lock attribution. The converged shape bundler itself writes after an install from the redirected Gemfile needs three lock edits the rewriter never made: the spec entry's GEM section, the DEPENDENCIES source pin, and (under a rotated grant) the lock remote.Fix
New
converge_gem_lock_source(mod.rs:3103): for each redirected dep on a CHECKSUMS-era lock,GEMsection into a patch-registryGEMsection (remote: <index-url>) — ledger editredirect_gemfile_lock_gem_source, original = upstream remote URL;DEPENDENCIESwith bundler's source-pin spelling<name> (= <version>)!(added in sorted position for transitive deps) —redirect_gemfile_lock_dependency_pin;redirect_gemfile_lock_source_url(mirrors the Gemfile refresh from the base PR).The result is byte-identical to what bundler writes itself, verified frozen-installable on bundler 4.0.15. Convergence is fail-soft: spec absent/duplicated, a legacy multi-remote GEM section, no DEPENDENCIES section, or (finisher commit) a hand-edited lock ordering DEPENDENCIES before the GEM sections all leave today's mixed state + warning untouched.
Behavior changes reviewers should know
redirect_gem_frozen_installno longer fires when the lock converges (CHECKSUMS-era) or when no lock exists — only on genuinely mixed pre-CHECKSUMS pairs.original(revert = restore Gemfile original + unfrozen install, same as today's documented manual path).Campaign repro
Hosted+vendored real-data sweep,
hosted-b2/hosted-b4checksums-lock oracles: exit-37 unfrozen and exit-16 frozen against the real bundler-4 CHECKSUMS lock. Note for lane re-runs: the campaignhosted-*/checksums-lockrun.shscripts PIN the old defect — on the fixed binary they abort loudly at "missing frozen-install warning" / "KNOWN-LIMITATION CANARY FLIPPED: unfrozen install SUCCEEDED". That abort IS the fix signature; the scripts need the same flip the in-repo canary got.Tests (red → green)
212232b, test-only): 4 units failed showing the mixed lock (GEM section stillremote: https://rubygems.org/, DEPENDENCIES unpinned); flipped e2e canary failed at the converged-GEM-section assert against the real bundler-4.0.15 CHECKSUMS lock.bdd8ba2): core lib green; e2e 5/5 — the renamed/flipped canarygem_hosted_checksums_lock_converges_and_installs_frozen_and_unfrozenproves a FROZEN (BUNDLE_FROZEN=true) fresh install succeeds with the lock byte-identical (exit-16 two-step gone) AND an unfrozen fresh install succeeds (exit 37 gone), both installing byte-verified patched gems. Unit coverage: full-file converged assert + ledger edits, converged-lock rerun no-op, rotated-grant lock-remote refresh, transitive dep sorted pin, CRLF lock convergence;gem_redirect_warns_about_frozen_installsnow pins the warning on the pre-CHECKSUMS mixed pair only.613f865scoped rustfmt;dc4c3aered — hand-edited lock with DEPENDENCIES before GEM must fail soft, not converge on stale indices;f5ddcb6the one-line ordering guard routing that shape to the mixed fail-soft path.Final gate at head: core lib 2273 passed / 0 failed;
e2e_redirect_gem_build -- --ignored5/5 vs real host bundler 4.0.15; adjacent suitesin_process_redirect(37) +e2e_vex_redirect(5) green; clippy on both crates — zero warnings in touched files (5 pre-existing in untouched test files, present on main).Reviewer sign-off
Adversarial review approved, red→green independently reproduced at unit and e2e level, including the frozen lock-byte-identity leg. The reviewer's main correctness challenge — the
(= version)!pin mismatching loose Gemfile requirements under frozen mode — dissolved on code read: the rewriter already rewrites the declared gem line to the exact version inside the source block, so the pin is exactly the spelling bundler writes for the post-redirect Gemfile. Disclosed gaps riding this PR: in-container docker-matrix legs (bundler 1.17/2.7/4.0.18) not re-run (disk rule; no images built); the empty-upstream-GEM-section shape (sole redirected dep) is unit-asserted but not bundler-verified end-to-end — worth one docker-matrix leg when images can be rebuilt; rotation x CHECKSUMS is covered at unit level only (the rotation e2e capstone runs with a pre-CHECKSUMS lock).Cross-repo follow-up
The depscan TS twin (
registry-rewritegem.ts) must be ported to match: the fully converged CHECKSUMS lock — GEM-section move +<name> (= <ver>)!pin + lock-remote rotation refresh — plus golden fixtures mirroring the new units (shared golden suite).🤖 Generated with Claude Code
Note
Medium Risk
Changes hosted gem lock rewriting and bundler install behavior for CHECKSUMS locks; incorrect convergence could corrupt locks or break frozen installs, but fail-soft paths and broad unit/e2e coverage limit blast radius.
Overview
Gem redirect rewrite now fully converges CHECKSUMS-era
Gemfile.lockfiles (bundler ≥ 2.6 / 4 default), not only the Gemfile source block and CHECKSUMS sha. Newconverge_gem_lock_sourcemoves the redirected gem’s spec into a patch-registryGEMsection, adds or updates theDEPENDENCIESsource pin (<name> (= <version>)!), and refreshes the lock remote in place when the grant token rotates—matching what bundler writes after a successful install.Warnings and ledger:
redirect_gem_frozen_installis emitted only when the pair stays mixed (no CHECKSUMS section or convergence refused); converged CHECKSUMS locks no longer get that caveat. New ledger edit kinds:redirect_gemfile_lock_gem_source,redirect_gemfile_lock_dependency_pin,redirect_gemfile_lock_source_url.Tests: Unit coverage for convergence, re-run no-op, rotation, transitive sorted pin, fail-soft when DEPENDENCIES precedes GEM, and CRLF locks. The e2e canary flips from pinning exit 37 / mixed-state failure to asserting frozen (
BUNDLE_FROZEN=true, lock unchanged) and unfrozen fresh-checkout installs both succeed with patched bytes.Reviewed by Cursor Bugbot for commit f5ddcb6. Configure here.