diff --git a/crates/socket-patch-cli/CLI_CONTRACT.md b/crates/socket-patch-cli/CLI_CONTRACT.md index 70f83ee43..32eb9706f 100644 --- a/crates/socket-patch-cli/CLI_CONTRACT.md +++ b/crates/socket-patch-cli/CLI_CONTRACT.md @@ -437,7 +437,7 @@ Recognition rules that hold for every ecosystem: * **Patch hosts.** A hosted reference counts only on `https://patch.socket.dev` or the `--patch-server-url` / `SOCKET_PATCH_SERVER_URL` origin, with no userinfo. The uuid is the URL's LAST canonical-uuid path segment, because grant tokens may themselves be uuid-shaped. The Go module prefix is fixed. `socket-patch-` registry / repository / source names count only through a pin. For a URL on any other host, see **Patch hosts** above. * **Pins, not definitions.** A registry, index or source *definition* alone (cargo `[registries]`, nuget ``, pom ``, uv index tables, `.npmrc`) never makes a reference, because it survives a reverted pin. Sections the package manager ignores are not read: npm's v2 `dependencies` mirror, a `.cargo/config.toml` shadowed by `.cargo/config`. A Socket pin inside a maven `` is diagnosed, never a reference. -* **Contested locks.** When one lock wires a package to a patch and another lock resolves the same `name@version` from a non-Socket source, the build's bytes depend on which package manager runs. The reference is then dropped with a `patched_ref_unattributable` diagnostic naming both files. This applies across npm / pnpm / yarn / bun and across uv / pylock / poetry / pdm / Pipfile.lock / requirements. PEP 723 script locks neither contest nor are contested. The root `requirements.txt` and its in-root `-r` includes count as one lock here: pip reads them as one requirement set, where a direct reference wins over a compatible `==` pin of the same version in another file of the set (#1086). A **bundled** npm copy (`inBundle: true`, or v1 `bundled: true`) of the same `name@version` contests the reference too, in the same lock, in the other npm lock of a shrinkwrap/package-lock pair, or in any other lock. npm unpacks it from the parent package's tarball, so no rewire reaches it and it stays unpatched. Bun and vlt unpack bundled copies the same way (#469, #471). pnpm unpacks bundled copies too, but its lock cannot tie one to a reference (see **Unattested references** below). For Bun, that is a `bun.lock` entry whose meta is `{ "bundled": true }`, or a `bun.lockb` record that a dependency edge with the `bundled` behavior bit reaches, including one Bun shares with a regular install. Such an entry is never a reference, and it contests the reference the same way. For vlt, the lock records no node for a bundled copy, so the copy is found in the installed store: a real package directory inside a store package's own `node_modules`. Hosted and vendored scans skip these copies with `redirect_bun_bundled_instance_skipped` / `redirect_vlt_bundled_instance_skipped` / `vendor_bundled_instance_skipped`. When a bundled copy is the only instance, vendoring refuses with `vendor_lock_entry_not_rewritable`. A reference withheld only because such an unreachable copy installs beside it (a bundled npm / Bun / vlt copy, an npm copy beneath a `hasShrinkwrap` package (#753) or one npm installs from a git / url / `file:` spec, or a yarn classic git or `file:` directory block of the same `name@version`) is still the rewriter's own wiring of exactly one package version: it is never attested, but `rollback`, `remove`, `list` and the hosted → vendored takeover treat it as an ordinary hosted pin and restore its upstream registry entry (#828), rather than refusing it with `hosted_wiring_contested`. Another entry of the **same** npm, Bun or yarn lock that resolves the wired `name@version` from a non-Socket source (for example a workspace member added after the rewire, then `npm install` / `bun install` / `yarn install`; for yarn berry, a registry locator such as `left-pad@npm:1.3.0` beside the hosted `…::__archiveUrl=` one) contests the reference too (#588). The package manager installs both entries, and that copy stays unpatched. Re-running `scan` / `vendor` rewires every copy. +* **Contested locks.** When one lock wires a package to a patch and another lock resolves the same `name@version` from a non-Socket source, the build's bytes depend on which package manager runs. The reference is then dropped with a `patched_ref_unattributable` diagnostic naming both files. This applies across npm / pnpm / yarn / bun and across uv / pylock / poetry / pdm / Pipfile.lock / requirements. PEP 723 script locks neither contest nor are contested. The root `requirements.txt` and its in-root `-r` includes count as one lock here: pip reads them as one requirement set, where a direct reference wins over a compatible `==` pin of the same version in another file of the set (#1086). A PyPI version matches under PEP 440 equality, so a requirements line pinned in any spelling that can only install the wired release (`==1.2.3.0`, `==v1.2.3`, `===1.2.3`) contests it like `==1.2.3` does; a range or wildcard (`~=1.2.3`, `==1.2.*`) does not (#1602). A **bundled** npm copy (`inBundle: true`, or v1 `bundled: true`) of the same `name@version` contests the reference too, in the same lock, in the other npm lock of a shrinkwrap/package-lock pair, or in any other lock. npm unpacks it from the parent package's tarball, so no rewire reaches it and it stays unpatched. Bun and vlt unpack bundled copies the same way (#469, #471). pnpm unpacks bundled copies too, but its lock cannot tie one to a reference (see **Unattested references** below). For Bun, that is a `bun.lock` entry whose meta is `{ "bundled": true }`, or a `bun.lockb` record that a dependency edge with the `bundled` behavior bit reaches, including one Bun shares with a regular install. Such an entry is never a reference, and it contests the reference the same way. For vlt, the lock records no node for a bundled copy, so the copy is found in the installed store: a real package directory inside a store package's own `node_modules`. Hosted and vendored scans skip these copies with `redirect_bun_bundled_instance_skipped` / `redirect_vlt_bundled_instance_skipped` / `vendor_bundled_instance_skipped`. When a bundled copy is the only instance, vendoring refuses with `vendor_lock_entry_not_rewritable`. A reference withheld only because such an unreachable copy installs beside it (a bundled npm / Bun / vlt copy, an npm copy beneath a `hasShrinkwrap` package (#753) or one npm installs from a git / url / `file:` spec, or a yarn classic git or `file:` directory block of the same `name@version`) is still the rewriter's own wiring of exactly one package version: it is never attested, but `rollback`, `remove`, `list` and the hosted → vendored takeover treat it as an ordinary hosted pin and restore its upstream registry entry (#828), rather than refusing it with `hosted_wiring_contested`. Another entry of the **same** npm, Bun or yarn lock that resolves the wired `name@version` from a non-Socket source (for example a workspace member added after the rewire, then `npm install` / `bun install` / `yarn install`; for yarn berry, a registry locator such as `left-pad@npm:1.3.0` beside the hosted `…::__archiveUrl=` one) contests the reference too (#588). The package manager installs both entries, and that copy stays unpatched. Re-running `scan` / `vendor` rewires every copy. * **Unattested references.** Some evidence shows a build may run a copy no wiring reaches, but cannot be tied to the reference's exact `name@version` or cannot say the build runs it. The reference then stays a reference: `list`, `rollback` and `remove` find it, and the ledgers' liveness gates (`vendor --check`, `scan`) keep treating the wiring as live, because re-running `scan` / `vendor` could never clear the evidence. Only `vex` omits it, as a run warning and as the `failed[].reason`. Four cases besides Gradle's (`vex_gradle_lock_above_base`): **pnpm bundled copies** (`vex_pnpm_bundled_copy`): a `packages:` entry's `bundledDependencies:` names the copies pnpm unpacks from that package's own tarball, but pnpm never locks them, so the bundled version is not in the lock. A pnpm reference whose package name a `bundledDependencies` list in the same lock names is omitted whatever its version (a missed attestation when the bundled copy is another version, never a false one), and `bundledDependencies: true` (or a value that cannot be read) omits every reference of that lock. It reaches no other lock. **deno.lock** (`vex_deno_lock_copy`): `deno install` installs a `package.json` project's npm dependencies from `deno.lock` and never reads `package-lock.json` / `pnpm-lock.yaml` / `yarn.lock`, so an npm-family reference whose `name@version` the `deno.lock` npm section also locks is omitted (#406). Whether Deno or another package manager populates `node_modules` (`nodeModulesDir: "manual"` allows either) is not in the files, so this holds whatever `nodeModulesDir` says. **Maven reactor roots** (`vex_maven_reactor_root`): a hosted pin in a root `pom.xml` that declares `` / `` is omitted, because a module's own literal `` overrides a root pin and module poms are not read (#261); hosted scan refuses such roots (`redirect_maven_multimodule_unsupported`), so this only covers pins an older release wrote. **Maven classifier copies** (`vex_maven_classifier_unpatched`): a `` dependency of a hosted-pinned GA is its own coordinate; the classifier-less pin does not retarget it and the grant serves only the main jar, so Maven fetches it from the next repository. When a live, non-profile classifier copy other than `sources` / `javadoc` (`tests`, a native or JDK-specific build) resolves any version other than the suffixed pin — its own literal ``, a `` entry with the same classifier, or no version in the root pom — the pin is omitted. `sources` / `javadoc` copies never run on a classpath and do not withhold the statement; hosted scan only warns about them (`redirect_maven_classifier_unsupported`) and skips the dep when an executable copy names the patched release. * **Shrinkwrap without a twin (#899).** npm >= 12 never reads `npm-shrinkwrap.json`: on a project whose only npm lock is the shrinkwrap it resolves the tree from the registry and writes a fresh `package-lock.json`, so a Socket reference there reaches npm <= 11 only. The reference is still discovered, so `list`, `rollback` and `remove` manage it, but `vex` omits the patch (`vex_npm_shrinkwrap_only`, as a run warning and as the `failed[].reason`). A `package-lock.json` beside it (wired the same way) makes it attestable again. Hosted and vendored runs still rewire the shrinkwrap and warn `redirect_npm_shrinkwrap_only` / `vendor_npm_shrinkwrap_only`. * **Lockless pins.** With no lock to name a version, a `Cargo.toml` pin (every declaration on `socket-patch-`, that registry defined on the patch host for the same uuid) or an exclusive nuget exact-id mapping is never a reference on its own, so v5.0 does not attest it (nor does `list` show it, or `rollback` / `remove` restore it — restore those files from version control). Only a pre-v5 redirect-ledger record naming a version the pin admits keeps it live. The same holds for a gem wired only in the `Gemfile` (the pre-bundler-2.6 mixed state, lock not converged). diff --git a/crates/socket-patch-cli/tests/e2e_vex_lockfile/pipenv.rs b/crates/socket-patch-cli/tests/e2e_vex_lockfile/pipenv.rs index f922cfb4e..f9bd4324a 100644 --- a/crates/socket-patch-cli/tests/e2e_vex_lockfile/pipenv.rs +++ b/crates/socket-patch-cli/tests/e2e_vex_lockfile/pipenv.rs @@ -367,3 +367,22 @@ fn requirements_direct_reference_contests_the_lock_wiring() { ], ); } + +/// #1602: pip installs the same upstream release for every spelling of an +/// exact pin, so each contests the Pipfile.lock wiring like `==1.2.3`. +#[test] +fn requirements_pin_spellings_contest_the_lock_wiring() { + contested_by_a_non_socket_direct_reference( + &flavors()[..1], + "requirements.txt", + &[ + ("arbitrary equality", format!("{NAME}==={VERSION}\n")), + ("trailing zero", format!("{NAME}=={VERSION}.0\n")), + ("v prefix", format!("{NAME}==v{VERSION}\n")), + ( + "padded, with a marker", + format!("{NAME}=={VERSION}.0.0 ; python_version >= '3'\n"), + ), + ], + ); +} diff --git a/crates/socket-patch-core/src/vendor/lock_inventory/pypi.rs b/crates/socket-patch-core/src/vendor/lock_inventory/pypi.rs index 95600fd18..cbe5232ff 100644 --- a/crates/socket-patch-core/src/vendor/lock_inventory/pypi.rs +++ b/crates/socket-patch-core/src/vendor/lock_inventory/pypi.rs @@ -670,7 +670,9 @@ async fn inventory_requirements_txt(view: &ProjectView<'_>) -> Option { diff --git a/crates/socket-patch-core/src/vex/discover/mod.rs b/crates/socket-patch-core/src/vex/discover/mod.rs index 0c02a4ee5..6ef311ece 100644 --- a/crates/socket-patch-core/src/vex/discover/mod.rs +++ b/crates/socket-patch-core/src/vex/discover/mod.rs @@ -460,13 +460,22 @@ pub struct ResolvedElsewhere { pub purl: String, /// Root-relative lock file. pub file: PathBuf, + /// The version matches only as spelled: a requirements `===X` pin, + /// which pip compares as a string + /// ([`Discovery::resolved_elsewhere_as_spelled`]). + pub as_spelled: bool, } impl ResolvedElsewhere { /// Whether this entry resolves the canonical base `purl`: the same - /// `name@version`, or any version of `name` for a versionless entry. + /// `name@version` (a PyPI version under PEP 440 equality unless the + /// entry matches only [as spelled](Self::as_spelled), so a hand-written + /// `==1.2.3.0` or `==v1.2.3` resolves `1.2.3`, #1602), or any version of + /// `name` for a versionless entry. fn covers(&self, purl: &str) -> bool { - self.purl == purl || purl_without_version(purl) == Some(self.purl.as_str()) + self.purl == purl + || purl_without_version(purl) == Some(self.purl.as_str()) + || (!self.as_spelled && same_pypi_release(&self.purl, purl)) } /// Whether the entry names no version, so may resolve any. @@ -475,6 +484,16 @@ impl ResolvedElsewhere { } } +/// Whether canonical bases `a` and `b` name one PyPI release under PEP 440 +/// equality (`@1.16`, `@1.16.0.0` and `@v1.16.0` are `@1.16.0`). Checks +/// the names first: the contest asks this for every pair of a ref and a +/// lock entry. +fn same_pypi_release(a: &str, b: &str) -> bool { + Ecosystem::from_purl(a) == Some(Ecosystem::Pypi) + && purl_without_version(a).is_some_and(|name| purl_without_version(b) == Some(name)) + && crate::utils::purl_key::pypi_same_release(a, b) +} + /// `pkg:/` of a versioned `pkg:/@`; /// `None` for a purl with no version (an npm scope's `@` is part of the /// name). @@ -926,6 +945,19 @@ impl Discovery { self.elsewhere.push(ResolvedElsewhere { purl: canonical_base_purl(&purl), file: PathBuf::from(file), + as_spelled: false, + }); + } + + /// [`Discovery::resolved_elsewhere`] for a version that matches only as + /// spelled: a requirements `===X` pin installs the release whose version + /// string is `X`, never a PEP 440-equal spelling of it (#1602). + pub(crate) fn resolved_elsewhere_as_spelled(&mut self, file: &str, purl: Option) { + let Some(purl) = purl else { return }; + self.elsewhere.push(ResolvedElsewhere { + purl: canonical_base_purl(&purl), + file: PathBuf::from(file), + as_spelled: true, }); } @@ -944,6 +976,7 @@ impl Discovery { self.elsewhere.push(ResolvedElsewhere { purl, file: PathBuf::from(file), + as_spelled: false, }); } @@ -1105,9 +1138,13 @@ impl Discovery { let tree = self.tree_of(&r.source_file); self.elsewhere.iter().find(|e| { let other = self.tree_of(&e.file); + // A tree that wires the release itself, in any + // spelling, is no contester. e.covers(&r.purl) && other != tree - && !wiring.contains(&(r.purl.clone(), other.to_path_buf())) + && !wiring.iter().any(|(purl, t)| { + t == other && (*purl == r.purl || same_pypi_release(purl, &r.purl)) + }) }) }) .flatten(); diff --git a/crates/socket-patch-core/src/vex/discover/pypi_other.rs b/crates/socket-patch-core/src/vex/discover/pypi_other.rs index c097ad022..a67b0a633 100644 --- a/crates/socket-patch-core/src/vex/discover/pypi_other.rs +++ b/crates/socket-patch-core/src/vex/discover/pypi_other.rs @@ -118,7 +118,7 @@ use crate::crawlers::python_crawler::canonicalize_pypi_name; use crate::utils::digest::sha256_hex; use crate::utils::hatch::dependency_specs; use crate::utils::requirements::{ - exact_pin, hash_options, logical_lines, split_comment, url_sha256_fragment, vendor_tag, + hash_options, logical_lines, parse_requirement, split_comment, url_sha256_fragment, vendor_tag, }; use crate::vendor::common::DeclTable; use crate::vendor::lock_inventory::pypi::{hosted_artifact_url, parse_pipfile_lock}; @@ -286,9 +286,9 @@ fn requirements_line(ctx: &DiscoverCtx<'_>, file: &str, line: &str, out: &mut Di return; } let Some((name, location)) = requirement_location(code) else { - // A registry requirement; an exact `name[extras]==X` pin is evidence - // against another lock's wiring of the same package. - out.resolved_elsewhere(file, registry_pin(code)); + // A registry requirement; a pin that can only install one release + // is evidence against another lock's wiring of that release. + registry_pin(file, code, out); return; }; let tag = comment.and_then(vendor_tag); @@ -342,11 +342,25 @@ fn egg_name(location: &str) -> Option<&str> { Some(&egg[..end]).filter(|name| !name.is_empty()) } -/// The `pypi_purl` of an exact `name[extras]==X` registry requirement -/// ([`exact_pin`]). -fn registry_pin(code: &str) -> Option { - let (name, version) = exact_pin(code)?; - pypi_purl(name, version) +/// Record a registry requirement that can only install one release as +/// evidence against another lock's wiring of it: an exact +/// `name[extras]==X` pin in any PEP 440 spelling (`==1.16.0.0`, +/// `==v1.16.0`; the contest compares it under PEP 440 equality), or an +/// arbitrary-equality `name===X` (compared as spelled, as pip does, #1602). +/// Ranges and wildcards (`~=1.16.0`, `==1.16.*`) are not evidence: what +/// they install depends on the index. +fn registry_pin(file: &str, code: &str, out: &mut Discovery) { + let Some(requirement) = parse_requirement(code) else { + return; + }; + if let Some(version) = requirement + .exact_version() + .filter(|version| crate::utils::pep440::is_valid(version)) + { + out.resolved_elsewhere(file, pypi_purl(requirement.name, version)); + } else if let Some(version) = requirement.arbitrary_version() { + out.resolved_elsewhere_as_spelled(file, pypi_purl(requirement.name, version)); + } } /// The `(declared name, location)` of a requirement that names an artifact: @@ -1770,6 +1784,100 @@ mod tests { } } + /// REGRESSION (#1602): pip installs the upstream release for every + /// spelling of a pin that can only select it (`===X`, a PEP 440-equal + /// `==X.0` or `==vX`, a shorter `==1.16`), so each contests another + /// lock's wiring of that release like the canonical `==X` does. A pin + /// of another release, a range or a wildcard contests nothing. + #[tokio::test] + async fn issue_1602_requirements_pin_spellings_contest_wiring() { + const WHL: &str = "six-1.16.0-py2.py3-none-any.whl"; + let vendored = vendored_wheel(UUID_A, WHL); + let wired = pipfile_lock(serde_json::json!({ "default": { + "six": { "file": format!("./{vendored}"), "hashes": [format!("sha256:{SHA}")] }, + }})); + let discover = |requirements: &'static str| { + let wired = wired.clone(); + async move { + let p = Project::new(); + p.write("Pipfile.lock", &wired) + .write("Pipfile", "[packages]\nsix = \"*\"\n") + .write("requirements.txt", requirements); + p.discover().await + } + }; + for line in [ + "six==1.16.0\n", + "six===1.16.0\n", + "six==1.16.0.0\n", + "six==v1.16.0\n", + "six==1.16\n", + "Six == V1.16.0.0 ; python_version >= \"3\"\n", + "six[extra]===1.16.0 --hash=sha256:abc\n", + ] { + let out = discover(line).await; + assert!(out.refs.is_empty(), "{line:?}: {:#?}", out.refs); + let c = out + .vendored_contest("pkg:pypi/six@1.16.0", UUID_A) + .unwrap_or_else(|| panic!("{line:?}: not contested")); + assert_eq!( + c.other, + std::path::Path::new("requirements.txt"), + "{line:?}" + ); + } + for line in [ + "six==1.16.1\n", + "six==1.16.0.1\n", + "six~=1.16.0\n", + "six==1.16.*\n", + "six===1.16.1\n", + // pip compares `===` as a string: no release is spelled `1.16`. + "six===1.16\n", + "other==1.16.0\n", + ] { + let out = discover(line).await; + assert_eq!(out.refs.len(), 1, "{line:?}: {:#?}", out.diagnostics); + } + } + + /// #1602: a lock tree that wires the release itself, under another + /// PEP 440 spelling, is not a contester even when it also pins it: + /// pip resolves the include's `==1.16.0` to the tree's own vendored + /// wheel (#1086), and Pipenv's `==1.16` entry is the same release. + #[tokio::test] + async fn issue_1602_a_tree_wiring_another_spelling_contests_nothing() { + let vendored = vendored_wheel(UUID_A, "six-1.16.0-py2.py3-none-any.whl"); + let short = vendored_wheel(UUID_A, "six-1.16-py2.py3-none-any.whl"); + let p = Project::new(); + p.write( + "Pipfile.lock", + &pipfile_lock(serde_json::json!({ "default": { "six": { + "file": format!("./{short}"), + "hashes": [format!("sha256:{SHA}")], + }}})), + ) + .write("Pipfile", "[packages]\nsix = \"*\"\n") + .write( + "requirements.txt", + &format!( + "-r base.txt\n./{vendored} --hash=sha256:{SHA} # socket-patch vendor: six==1.16.0\n" + ), + ) + .write("base.txt", "six==1.16.0\n"); + let out = p.discover().await; + assert!( + !out.diagnostics + .iter() + .any(|d| d.code == DIAG_REF_UNATTRIBUTABLE), + "{:#?}", + out.diagnostics + ); + let mut purls: Vec<&str> = out.refs.iter().map(|r| r.purl.as_str()).collect(); + purls.sort(); + assert_eq!(purls, ["pkg:pypi/six@1.16", "pkg:pypi/six@1.16.0"]); + } + /// REGRESSION (#1549): a non-Socket direct reference whose version /// can't be read (a source directory, a VCS url, an artifact named /// for no release) may install any version of its package, so it diff --git a/crates/socket-patch-core/src/vex/discover/testing/golden.rs b/crates/socket-patch-core/src/vex/discover/testing/golden.rs index d0ef62e6c..c18c68fe7 100644 --- a/crates/socket-patch-core/src/vex/discover/testing/golden.rs +++ b/crates/socket-patch-core/src/vex/discover/testing/golden.rs @@ -209,7 +209,7 @@ fn render(out: &Discovery, root: &Path) -> Value { let elsewhere_rendered: Vec = elsewhere .iter() .map(|e| { - let ResolvedElsewhere { purl, file } = e; + let ResolvedElsewhere { purl, file, .. } = e; json!({ "purl": purl, "file": path_str(file) }) }) .collect();