diff --git a/.agents/docs/2026-09-11-sdk-toolchains-and-ios-local-verification.md b/.agents/docs/2026-09-11-sdk-toolchains-and-ios-local-verification.md index f524df1fe..dc8537487 100644 --- a/.agents/docs/2026-09-11-sdk-toolchains-and-ios-local-verification.md +++ b/.agents/docs/2026-09-11-sdk-toolchains-and-ios-local-verification.md @@ -164,7 +164,7 @@ clang. It needs: | | comes from | why | |---|---|---| | the compiler | `xim:llvm` | any sufficiently new clang emits arm64 Mach-O for an iOS deployment target | -| the C++ runtime | the payload's libc++ | as on every other Apple row | +| the C++ runtime | the payload's libc++ | as on every other Apple row -- superseded: the payload's archives are macOS objects and the SDK's libc++ is another release; the rows take `llvm.libcxx` from the graph (2026-09-13 record for #630, §5) | | the **SDK** | the machine's Xcode | headers and stub libraries, not redistributable | | running on a simulator | the machine's `simctl` | a proprietary runtime that exists only on its own OS | diff --git a/.agents/docs/2026-09-13-630-what-a-framework-still-hits-in-the-engine.md b/.agents/docs/2026-09-13-630-what-a-framework-still-hits-in-the-engine.md new file mode 100644 index 000000000..8507750a6 --- /dev/null +++ b/.agents/docs/2026-09-13-630-what-a-framework-still-hits-in-the-engine.md @@ -0,0 +1,882 @@ +--- +subject: triage +status: landed +--- + +# What a framework and its ecosystem library still hit in the engine: the ten items of #630, read against the code + +**Status:** landed on 2026-09-13 as mcpp 2026.9.13.2 (mcpp-community/mcpp#631), +`llvm.libcxx` 22.1.8.1 (new repository `mcpplibs/libcxx`), +`llvm.compiler-rt-builtins` 22.1.8.5 (mcpplibs/compiler-rt-builtins#1, #2), +mcpp-index #408 and #409, and a `mcpp:plugins` change that follows the +release. Every statement about the engine was read at `b63dc4e5` (mcpp +2026.9.13.1), the version the issue measured against. §0 classifies the ten +items; §1 states the rules the changes share; §2 to §9 take the items one at +a time with the code, the classification, the shape and the criterion in both +directions; §10 is the order and what the ecosystem side deletes; §12 is the +task list; §13 records what landed and where each claim was measured. + +## 0. The ledger + +#630 lists eight items and restates two from #622. The issue's own framing is +correct in one respect that governs everything below: nothing in it is +framework-specific and nothing blocks the work, so each item is judged by +whether it is a property of the engine that any second framework would hit, +not by whether it inconveniences this one. + +| # | the issue says | measured | classification | +|---|---|---|---| +| 1 | two `git` revisions of one identity: the root wins silently | no field of a `git` or `path` declaration is ever compared; the record kept is the one dequeued first, and the root is dequeued first because it is seeded first (`prepare.cppm:6407-6432`, `6764-6774`) | engine defect, general. The rule the issue infers is an accident of queue order | +| 2 | `path` in the root against `git` in a dependency is a hard error | confirmed at `prepare.cppm:6480-6486`; no override table exists in the manifest | engine gap, general. Decided together with 1 | +| 3a | Mach-O programs are not staged, so a bundle carries no deployed files | the refusal at `pack.cppm:1240` precedes the first staging step at `pack.cppm:1279`; the pipeline already carries the failure to the dispatched format with an empty stage directory (`pipeline.cppm:352-365`, `408-410`) | engine defect: an order of operations. Declared files need no loader | +| 3b | the Mach-O closure needs a reader | `binfmt.cppm` has a static reader for ELF (`216-310`) and for PE (`320-415`); Mach-O is identified and refused (`472-497`). The ELF pack path still runs the loader (`pack.cppm:1376`) although the reader exists | engine gap, general. One rule for three formats | +| 4 | iOS links the payload's libc++ headers against the SDK's libc++, and no compiler-rt | by construction: the compile side emits `-nostdinc++ -isystem /include/c++/v1` (`hostflags.cppm:341-343`, `linkmodel.cppm:178`) while the link side chooses the SDK's `-lc++` (`distribution.cppm:485-530`). The 2026-09-11 record's table says "the payload's libc++" for the iOS rows; `distribution.cppm` says the opposite, and the code is right | engine defect: the two halves of one seam are chosen in two files. Resolved by a runtime package for Apple's other platforms, so that the iOS rows take the macOS rows' contract (§5) | +| 5 | `min_api_level` is honoured and reported as unsupported | confirmed (`toml.cppm:2635`, `2744-2760`); it is the only parsed non-table key absent from the known list, the drift dates from #610, and `--strict` promotes the warning to an error (`prepare.cppm:2058-2061`) | engine defect. Not cosmetic under `--strict` | +| 6 | a cached host tool outlives its source | the store key holds package, version, host, compiler, profile, features and `name@version` of the closure, and no source content (`tool_store.cppm:77-97`, `prepare.cppm:8721-8745`). The object cache excludes `path` and `git` packages by rule (`cache_key.cppm:44-49`); the tool store has no such rule. Measured on 2026-09-08 with `examples/12` | engine defect, known and unrecorded until now | +| 7 | `mcpp emit xpkg` reads the top-level `[xlings.workspace]` only | confirmed, and already reported by the `publish/target-axis-tools` warning (`publisher.cppm:215-233`). At build time the consumer's mcpp folds a dependency's target-axis tools and provisions them (`prepare.cppm:265-294`, `7009-7012`, `4541-4560`), so the consequence is a descriptor that under-states install-time dependencies, not a link failure | half usage (the warning names the workaround), half engine: a selector that names only an operating system is a platform | +| A8 | a dependency's `[xlings.workspace]` should reach the consumer's build program | decided in the 2026-09-12 record §2.8: a host module's declarations reach every build program compiled from it (`fillXpkgDirs`, `prepare.cppm:5680-5751`); an ordinary library's do not, by design | usage. Declined again, with the reason restated in §8 | +| A9 | one artifact from several targets | the library route already takes several triples (`build_and_pack_library`, `library_pipeline.cppm:113`); the program route refuses more than one (`cmd_publish.cppm:124-129`). On Android an `app` is a shared object | engine gap, small. The universal APK is the library route applied to an app | + +The issue's "what is not here" list is accepted as written. One addition: +HuxerUI's builtins lookup falls through to Xcode's `libclang_rt` through +`xcrun --find clang` when the payload lacks the archive. That is the host +fallthrough the recorded host-surface rule forbids, and it exists only because +the engine is silent where it should refuse (§5.3). It is listed in §10 as a +workaround the ecosystem deletes. + +## 1. The rules the proposals share + +Six items reduce to four rules, three of which the engine already applies +elsewhere. Naming them is what keeps the ten fixes from being ten policies. + +**Rule A. One identity, one resolution, said aloud.** The tool plane already +has this: `addrset::unify` (`address_set.cppm:124-177`) lets the declaration +nearest the artifact win, checks every other declaration against the winner, +reports a difference as `xlings/version-override` and refuses a violated +requirement. The dependency graph has the refusal for a violated SemVer +constraint and nothing else. Items 1 and 2 give it the rest of the same +rule. The precedent for "the root alone may decide a whole-graph property" is +`DependencySpec::linkage`, honoured only on the root's edges because "a form +is a whole-image decision" (`dep_spec.cppm`). + +**Rule B. A closure is read, not run.** PE already does this +(`pe_closure` through `needed_names`). Reading is the only mechanism that +works for a format whose loader ignores the trace variable (Mach-O), for a +host that cannot execute the artifact (a Linux host packing a macOS program; +a Windows host packing an ELF, `pack.cppm:1257`), and for an operating system +whose libraries are not on disk at all (macOS 11 and later keep them in the +shared cache). Items 3a and 3b apply it; the ELF path is the third +beneficiary and is not changed in this batch. + +**Rule C. The two halves of a seam are chosen in one place.** The libc++ a +translation unit is compiled against and the libc++ it is linked against are +one decision. Today the compile half is made in `hostflags.cppm` from a +question about the C ABI and the link half in `distribution.cppm` from a +question about the target's format and contract. Item 4 makes the two halves +agree by supplying the runtime the link side lacked, so that the contract the +macOS rows already have applies unchanged; the header set is then not a +second decision at all. + +**Rule D. A list that a check reads must be the list the parser reads.** The +`[target.]` sweep keeps its own array of known keys next to a parser +that finds keys by name; the two drifted in #610. Item 5 is one line today and +one table tomorrow. + +Two further rules bind single items. **A cache key holds everything that can +change the bytes** (item 6; `cache_key.cppm` states it and the tool store does +not follow it). **A selector that names only an operating system is a +platform** (item 7; the descriptor's three blocks are keyed by exactly that). + +## 2. Items 1 and 2: one identity, two declarations + +### 2.1 What the code does + +The worklist is seeded from the root's `[dependencies]`, `[dev-dependencies]` +and `[build-dependencies]` before any transitive edge is pushed +(`prepare.cppm:6407-6432`), and processed first in, first out. When an +identity is already in `resolved`: + +- a different *kind* (`path` / `git` / `version`) is refused with "requested + as both a … dep (by …) and a … dep (by …). Pick one" (`6480-6486`); +- two `version` declarations are AND-merged through `try_merge_semver` + (`6488`; `resolver.cppm:257-288`), and an irreconcilable pair is refused; +- two `git` or two `path` declarations fall through to the edge-recording + branch under the comment "same version (or compatible path/git)" + (`6764-6774`). `ResolvedRecord` (`4612-4626`) holds `version`, + `constraint`, `requestedBy`, `source` and `depIndex`; it holds no path and + no `gitRev`/`gitRefKind`, so there is nothing to compare. + +Therefore the rule is not "the root wins". It is "the first requester +dequeued wins", which is the root when the root declares the identity and an +arbitrary dependency otherwise. `mcpp.lock` records `spec.gitRev`, the literal +the root wrote, as `version` (`12112-12129`), and records root-declared git +dependencies only; the sha in the issue's lock excerpt is there because the +root wrote a sha. `docs/05` promises to answer "what happens when two of them +disagree" and answers it for SemVer only; no test exercises the "Pick one" +refusal. + +### 2.2 Decision + +Rule A, applied to the dependency graph, with the root's privilege bounded +the way `linkage` bounds it: + +| first declaration | second declaration | today | proposed | +|---|---|---|---| +| any | same kind, same reference | edge only | unchanged | +| `version` | `version`, different constraint | SemVer merge, refuse if empty | unchanged | +| `git` | `git`, different `rev`/`tag`/`branch` | silent, first dequeued wins | the root's declaration wins if the root is a requester, else the first dequeued; `dependency/source-override` warning naming both requesters and both references | +| `path` | `path`, different directory | silent | same as the row above | +| root `path` or `git` | dependency `git` or `version` | refused | the root's wins, same warning; a dependency's `version` constraint is checked against the `[package] version` of the root's checkout and refused if violated | +| dependency `path`/`git` | dependency of another kind | refused | unchanged: a dependency may not change another dependency's source kind. The hint gains one sentence: "declare `` in the root to settle it" | + +The check in the fifth row is what makes the issue's "where this is going" +paragraph unnecessary as a separate feature: once the framework is on the +index with SemVer tags, a library's `^0.3` is a requirement and the root's pin +(a version, a tag, a path) is checked against it exactly as `unify`'s +`Violated` verdict checks a tool pin against a floor. There is no solver and +no second policy; the checkout's manifest already states its version. + +A `[patch]` table is not proposed. The root-wins rule is the general form for +a graph whose root is the artifact; Cargo needs `[patch]` because its +resolver may select several versions of one crate and the root's own edge is +not privileged. mcpp has one package per identity by decision (the 2026-09-06 +record) and the root's edge is already privileged for `linkage`. + +### 2.3 Shape + +- `ResolvedRecord` gains `sourceRef` (for `git`: `#=`; for + `path`: the canonical absolute directory) and `fromRoot` (the requester's + `consumerDepIndex == kMainConsumer`). +- At the `resolved.find(key)` hit, the six rows above are one `switch` over + (kind of the record, kind of the item, `fromRoot` of each). A root override + re-points `depIndex` when the record came from a dependency and the root + arrives later; this cannot happen under FIFO seeding but is written so that + the rule does not depend on the queue order that produced today's accident. +- Warnings go through `mcpp::diag::warning("dependency/source-override", …)` + with the same three-part shape the tool plane uses: what was declared by + whom, which won and why, how to take the other. +- `docs/05` gains the table above under a heading that answers the sentence + its own preamble promises; the zh mirror gains the same table. + +### 2.4 Criteria + +One e2e fixture with an application and two libraries, all `path`-local git +repositories so the test needs no network: + +1. root `git rev=A`, library `git rev=B`: exit 0; the warning names the root, + the library, `A` and `B`; `mcpp.lock` records `A`; the library's object + is compiled against `A` (a marker header present only at `A`). +2. root `path`, library `git`: exit 0 with the same warning; the compiled + library sees the working tree's marker, not the committed one. +3. the root's declaration removed, two libraries at `B` and `C`: exit 0; the + warning names the two libraries; the winner is stated in the message. +4. two libraries, one `path` one `git`: refused with today's message plus + the new hint. +5. root pins a checkout whose `[package] version` is `0.2.0`; a library + requires `>=0.3`: refused, naming both. +6. Negative direction: the same reference declared twice produces no + warning, and `mcpp build --strict` exits 0. Without this row the fixture + passes on an implementation that warns unconditionally. + +## 3. Item 3a: stage what is declared before walking what is discovered + +### 3.1 What the code does + +`pack::run` refuses a Mach-O program on every host at `pack.cppm:1240-1255`; +the staging directory is created at `1279`, the program copied at +`1286-1295`, and `stage_runtime_files` (the `mcpp::deploy` and +`[runtime] deploy` entries, collected into `plan.runtimeDeployFiles`) runs at +`1297`. The pipeline already tolerates the refusal for a dispatched format: +it prints a warning, leaves `pack_stage_dir` empty, and lets +`${mcpp.stage_dir}` refuse at expansion with the reason attached +(`pipeline.cppm:352-365`, `408-410`). The comment there records the decision: +"staging is a service to the provider, not a precondition for dispatch". + +So `dist-apple` reaches its action and finds nothing to place, which is what +the issue measured. The deployed files are declared, not discovered; the +loader is needed for none of them. + +### 3.2 Decision + +The staged tree is produced in every case and the closure is one step within +it whose outcome is recorded: + +1. wipe and create the staging root; +2. copy the program (or the shared object, on the Android rows); +3. stage the declared files (`stage_runtime_files`); +4. resolve the closure by the format's mechanism; a format whose mechanism is + unavailable returns a reason instead of a list; +5. the format-specific tail (rpath rewrite, strip, debug split) runs only when + step 4 produced a list. + +For `--format tar` and `--format dir` an unavailable closure remains the +command failing, as today: the archive is the closure. For a dispatched +format the tree from steps 1 to 3 is handed over, `pack_stage_dir` is set, +and the stage manifest gains `closure = "walked" | "not-walked"` with the +reason. The warning the pipeline prints today changes from "no staged tree" +to "staged without its dependency closure: ", and a provider that +needs the closure reads the manifest and says so. + +This is the smallest change that gives `dist-apple` a resource destination to +fill (`Contents/Resources/` on macOS, the bundle root on iOS; that placement +is the member's, as the 2026-09-12 record decided for `dist-apk`). It also +removes the platform-specific refusal from the engine's staging path: the +refusal becomes step 4's reason for one format on one host. + +### 3.3 Criteria + +- On a macOS runner, `mcpp pack --format app` of a program with one + `mcpp::deploy` entry: the staged tree contains `bin/` and the + deployed file; the stage manifest says `not-walked`; the `.app` the member + produces contains the deployed file at the member's destination. +- On a Linux host, `mcpp pack --format dir` of an ELF program: byte-identical + tree to today's (the reorder must not change the ELF product). +- Negative direction: `mcpp pack --format tar` of a Mach-O program still + exits non-zero with the reason; without this row the change could be read + as "tar of an incomplete closure succeeds". + +## 4. Item 3b: a Mach-O reader, and the rule it completes + +### 4.1 What exists + +`binfmt.cppm` reads `DT_NEEDED` from an ELF by walking `PT_DYNAMIC` and +`DT_STRTAB` (`216-310`) and the import directories of a PE (`320-415`); +`needed_names` dispatches on the magic (`492`) and returns "not implemented" +for Mach-O (`494-497`). The pack path uses the PE reader for PE closures +(`pack.cppm:908`) and `ldd_parse`, which runs the program under +`LD_TRACE_LOADED_OBJECTS=1` (`548-561`), for ELF (`1376`). Mach-O is +identified in all five magics including the fat wrapper (`472-481`). + +### 4.2 Decision + +Complete `needed_names` for Mach-O and make the closure walk a function of +the format's reader (Rule B): + +- **Reading.** For a thin file, walk the load commands; `LC_LOAD_DYLIB`, + `LC_LOAD_WEAK_DYLIB` and `LC_REEXPORT_DYLIB` contribute names, + `LC_RPATH` contributes search entries. For a fat file, select the slice + whose `cputype`/`cpusubtype` match the resolved triple and read that; a + slice for another architecture is not the artifact being packed. +- **Resolving.** `@executable_path` and `@loader_path` are resolved against + the staged program; `@rpath` against each `LC_RPATH` entry in order, with + the same two prefixes substituted. An absolute name under `/usr/lib/` or + `/System/Library/` is the operating system's and is never bundled: the + Mach-O row of `is_system_lib`, alongside the glibc row that exists for + ELF (`pack.cppm:529-533`). The result is the `ResolvedDep` list the ELF + branch already consumes. +- **Bundling.** The dylibs that resolve to a payload or a build-tree path are + copied beside the program, and the program's `LC_RPATH` must then name the + bundled directory. This is the Mach-O counterpart of the `$ORIGIN` rewrite + and it needs a load-command editor (`LC_RPATH` entries live inside the + header's padding, and a longer entry may not fit). The reader and the + bundle-set computation land first and are measured on a macOS runner; the + rewrite is designed after that measurement, not before it, since the + alternative (linking with `-rpath @executable_path/../Frameworks` at build + time so that no rewrite is needed) may make the editor unnecessary for the + artefacts mcpp produces. + +The ELF path is not moved off `ldd_parse` in this batch. It is noted as the +third beneficiary: the reader exists, and reading is what would let a Linux +host pack for another glibc or a Windows host pack an ELF. + +**What landed.** The reader (`macho_needed`, thin and fat, both byte +orders), the Mach-O row of `is_system_lib`, the `@rpath` resolver +(`resolve_macho_names`) and `needed_names`'s dispatch to them, with unit +tests over generated fixtures. The closure step of `pack::run` still reports +a Mach-O program as `not-walked`: wiring the reader into that step without +bundling would name a closure the tree does not carry, and bundling needs the +`LC_RPATH` decision the measurement above is for. The dispatched format +therefore receives the program and the declared files (item 3a) and a +manifest that says so, which is what `dist-apple` needs today. + +### 4.3 Criteria + +- Unit: a checked-in thin arm64 Mach-O and a fat (`x86_64` + `arm64`) Mach-O + with known `LC_LOAD_DYLIB` and `LC_RPATH` lists; `needed_names` returns + them in order; the wrong slice is never read (a name present only in the + other slice is absent). +- On a macOS runner: a program linked against the payload's `libc++.dylib` + through `@rpath` resolves it to the payload path and lists + `/usr/lib/libSystem.B.dylib` as the system's; the staged tree contains the + former and not the latter. +- Negative direction: a program with no `LC_RPATH` and an `@rpath` name + resolves to "unresolved: " and the pack reports it, rather than + silently skipping the entry. + +## 5. Item 4: one libc++ for the iOS rows, `import std` kept, and a refusal where the driver is silent + +### 5.1 What the code does + +On the iOS rows `cAbiPrebuilt` is true (the C library is the SDK's), so +`graphSuppliesTarget` is false (`hostflags.cppm:341`) and the compile side +emits the payload's header set: `--no-default-config -nostdinc++ +-isystem /include/c++/v1` (`linkmodel.cppm:178`), followed by +`-isysroot ` (`hostflags.cppm:402-403`). The std module is precompiled +from the payload's `share/libc++/v1/std.cppm` against the same headers with +`-isysroot ` added (`prepare.cppm:3768-3790`). The link side, in +`distribution.cppm:485-530`, chooses `HostCoupled` with ` -lc++` for every +Apple cross target, under a comment that states the reason with precision: +"iOS takes its C++ runtime from the SDK, and has no other option": the +payload's static archives are built for macOS and ld64 refuses them in an +iOS link, and the payload's `libc++.dylib` is not on a device. + +So every translation unit, and the std module, is compiled against libc++ +22's headers and linked against the libc++ the SDK's `libc++.tbd` describes. +The mismatch is silent until an inline function in the newer headers +references a symbol the older dylib does not export, which is the issue's +`__hash_memory` and `__atomic_notify_all_global_table`. The 2026-09-11 +record's table for item D lists "the C++ runtime: the payload's libc++, as +on every other Apple row"; that statement was superseded by +`distribution.cppm` during the same batch and the record was not corrected. +The row's workaround (pin `llvm@20.1.7`) works because libc++ 20's headers +happen to reference nothing the iOS 18 SDK lacks; it is a coincidence about +two version numbers, not a fix. + +The same class exists on macOS: the `HostCoupled` fallback (no deployment +floor resolved, or a payload without archives; `distribution.cppm:542-551`) +links `/usr/lib/libc++` under the payload's headers. + +**Measured on a macOS runner (Xcode 16.4, run 34757885971, 2026-09-13), +while this revision was written.** The payload's `lib/clang/22/lib/darwin/` +holds `libclang_rt.osx.a` and the macOS sanitizer runtimes and no `ios` or +`iossim` archive; `clang -###` for `arm64-apple-ios18.0-simulator` adds no +`libclang_rt` at all. The payload's libc++ is `_LIBCPP_VERSION 220108`; the +macOS 15.5 and iOS 18.5 SDKs carry `190102`, their `libc++.tbd` exports +neither `__hash_memory` nor `__atomic_notify_all_global_table`, and neither +SDK ships `usr/share/libc++/v1` (no module sources). A program using +`std::unordered_map` and `std::atomic::notify_all` +failed to link in all three arrangements tried: payload headers over the +SDK's dylib (mcpp's shape today), SDK headers over the SDK's dylib, and +payload headers over the payload's macOS archives. The issue's report is +therefore not a property of one application: with this payload, every iOS +program that reaches those inline paths fails at link. + +### 5.2 The constraint, and the mechanism the engine already has + +Rule C says the headers, the module and the runtime are one libc++. The +comment in `distribution.cppm` says the runtime can only be the SDK's. Both +cannot hold while `import std` is to be kept, because the SDK's libc++ is a +version the payload carries no module sources for. The first draft of this +record accepted the comment and moved the headers to the SDK, leaving the +std module to a measurement of whether the SDK ships `std.cppm`. That was +wrong in its premise: "no other option" is a statement about the *payload as +published*, not about what the dependency graph can supply, and the engine +already has the mechanism for a graph package to be the C++ runtime beneath +the compiler. + +`mcpplibs/openkal-llvm-runtime` is that mechanism in use: libc++, libc++abi +and libunwind as a source package, configured for openkal-musl, declaring +`provides = ["hosted-standard-library", "mcpp:c++-abi=libc++", +"mcpp:compiler-runtime=compiler-rt"]`, `std-module = "…/std.cppm"`, +`std-compat-module` and `std-module-flags`. The engine resolves the five +layers of the target side independently (`targetside_model.cppm:492-563`; +`TargetSide{compiler, compilerRuntime, kernelAbi, cAbi, cxx}`), broadcasts a +graph-origin layer's include directories into every package's private build +(`prepare.cppm:9563-9617`, `note_layer(CxxAbi, …)`), adopts the package's +std module over the toolchain's (`prepare.cppm:10362-10406`), and resolves +the distribution contract to `SelfContained` with `-nostdlib++` whenever the +C++ runtime is the graph's (`distribution.cppm:455-477`). The same code +serves the bare rows, where `mcpplibs/picolibc` is the C library and +`llvm.compiler-rt-builtins` the compiler runtime, both from source, both +compiled with the consuming program's own flags. + +What keeps that mechanism from serving the iOS rows is one proxy. Four +sites ask "does the payload's C++ runtime apply" and answer it with the +C-library question `cAbi.prebuilt()`: + +| site | what it decides | reads | +|---|---|---| +| `hostflags.cppm:341` | whether the payload's libc++ `-isystem` block is emitted | `!cAbiPrebuilt` | +| `flags.cppm:616` | the same, on the compile-flag builder | `cAbi.prebuilt()` | +| `flags.cppm:660-694` | whether the payload's link-side driver flags are emitted | `cAbi.prebuilt()` | +| `flags.cppm:1166` | `graphCxxRuntime`, the input of the contract's early `SelfContained` branch | `!cAbi.prebuilt()` | + +The two questions coincide for openkal (both layers from the graph) and for +a native build (both from the payload). They come apart on the iOS rows, +where the C library is the located SDK (`Origin::Payload`, prebuilt) and the +C++ runtime can be a graph package. `check_layering` +(`targetside_model.cppm:584-600`) already refuses the opposite pairing, a +payload C++ runtime over a graph C library, so `cxx.fromGraph()` is the +question, and it is already computed. + +### 5.3 Decision + +**The C++ runtime of the iOS rows is a graph package, `llvm.libcxx`, and +the engine asks the C++ layer's own question at the four sites.** The +package is libc++ and libc++abi from `llvmorg-22.1.8` as source, with a +generated `__config_site`, `__assertion_handler`, `std.cppm` and +`std.compat.cppm`, under the namespace and version rule the +`llvm.compiler-rt-builtins` package established (upstream's namespace, +upstream's version, a fourth segment for the packaging). It is not +Apple-specific: libc++ recognises Darwin and glibc by itself, so one +configuration serves a hosted target whose payload cannot supply a static +libc++ for it, which today is the three iOS rows and tomorrow may be others. +libunwind is not carried: on Apple platforms the unwinder is libSystem's, +and on Linux the payload's is linked by the driver. + +A framework declares it once, under the rows that need it, and every +application inherits it through the ordinary dependency edge: + +```toml +[target.'cfg(os = "ios")'.dependencies] +llvm.libcxx = "22.1.8.1" +llvm.compiler-rt-builtins = "22.1.8.5" +``` + +An application without a framework writes the same three lines. The two +packages stay separate for the reason the builtins package itself records: +a C program over the same rows needs the second and not the first, and one +edge per layer is what lets either be replaced. This is the +openkal shape (a program declares `openkal-llvm-runtime`) and the shape the +bare rows have for `compiler-rt-builtins`; the engine injects no dependency +and the row table names none. The alternatives and why they are not taken: + +- **A prebuilt payload built on a macOS runner** (`xim:llvm-apple-runtimes`, + the shape of this record's first revision). It needs a build farm the + ecosystem does not have, one archive per Apple platform and architecture, + and a version rule tying the payload to the archives. A source package + needs none of these and is what the engine already consumes. +- **The SDK's libc++ with module sources of the SDK's version.** It ties + every object to the machine's Xcode, adds a version-keyed package family + that has to follow Apple's releases, and puts the C++ runtime on Apple's + side of the seam the 2026-09-11 record drew ("the compiler is ours; only + the SDK is Apple's"). It remains the shape of an explicit + `cxx_runtime = "host-coupled"` request and is not built here. +- **Pinning the payload's libc++ to the SDK's.** A payload cannot know which + SDK a machine has. The `llvm@20.1.7` workaround is this alternative done by + hand, and it holds only until the next SDK. + +**What the engine does when no package is declared.** The runtime is the +SDK's. A graph that does not import `std` takes the SDK's headers +(`-nostdinc++ -isystem /usr/include/c++/v1`; clang's Darwin driver +would otherwise prefer the libc++ installed beside the compiler): one libc++ +on every line, which Rule C asks for. A graph that imports `std` keeps the +payload's module and headers over the SDK's dylib, which is what every iOS +build got before this batch; the SDKs ship no module sources (measured +above) and the engine does not consume one, so there is no consistent pair +to switch to. That pairing is reported once as a degradation +(`target/cxx-runtime`) naming the hazard and the two package lines, and the +build proceeds. A refusal was written first and withdrawn on review: it +would have broken a program that built the day before, while the +degradation names the remedy at the first build and costs nothing until an +inline path reaches an export the older dylib lacks. + +The engine change, in full: + +1. `HostFlagOptions` gains `cxxFromGraph`, read from + `plan.targetSide.cxx.fromGraph()`. The payload's libc++ `-isystem` block + is withheld when it is true, and `-nostdinc++` is emitted so that the + driver's own C++ search (the payload's headers beside the compiler, or the + SDK's) contributes nothing; the package's headers arrive through the + layer broadcast that already exists. +2. `flags.cppm:1166` reads `cxx.fromGraph()`. The contract's early branch + then returns `SelfContained` with `-nostdlib++` for an iOS row that + declares the package, before the Apple-cross branch is reached. That + branch stays for the no-package case and its comment is corrected: the + SDK is the only runtime *the payload* can offer. +3. `flags.cppm:660-694` is unchanged: the link-side driver flags are the C + library's business, and `-stdlib=libc++` beside `-nostdlib++` is inert. +4. `graph_runtime_compile_flags` (`model.cppm:491-530`) emits + `-femulated-tls` on Mach-O only when the C library is the graph's. It was + written for openkal-macos, which has no dynamic loader to bootstrap a + thread-local; iOS has one. The visibility flags stay for every Mach-O, + since a hidden copy is what keeps dyld from unifying it with the system + libc++ that UIKit loads (the #117 forensics). +5. The std-module adoption at `prepare.cppm:10362` accepts + `mcpp:c++-abi=libc++` as the current spelling of `hosted-standard-library` + and continues to accept the older one. +6. On an Apple cross target without a graph C++ runtime: when the graph + does not import `std`, `Toolchain::appleSdkCxxHeaders` is set and the + compile side emits `-nostdinc++ -isystem /usr/include/c++/v1`; when + it does, the payload's module stays and prepare reports the + `target/cxx-runtime` degradation naming `llvm.libcxx` and + `llvm.compiler-rt-builtins`. +7. The builtins archive. Clang's Darwin driver adds + `libclang_rt..a` from its own resource directory and, when the + file is absent, continues without it (its source says missing runtime + libraries are tolerated so that a build without compiler-rt can proceed); + that is the issue's `__isPlatformVersionAtLeast` undefined at link with no + earlier message. The measurement above settles which half owns the fix: + the 22.1.8 payload ships no iOS archive, so the compiler runtime of the + iOS rows is a graph package as it is on the bare rows. + `llvm.compiler-rt-builtins` gains an Apple source selection + (`os_version_check.c`, which is where `__isPlatformVersionAtLeast` lives, + and the generic routines for `aarch64` and `x86_64`; the exclusions the + openkal package measured for Mach-O), and the framework declares it + beside `llvm.libcxx`. The engine's part is the layer model's: on an Apple + cross target whose payload carries no archive for the platform and whose + graph declares no `mcpp:compiler-runtime`, `TargetSide::compilerRuntime` + is unsupplied, and prepare reports it once as a degradation naming the + platform, the missing file and the package that supplies it. A refusal + was considered and rejected: a program that never reaches an availability + check links and runs today, and refusing it would trade a diagnosed hazard + for a regression. The engine never looks in Xcode for the archive. +8. The per-row `toolchain = "llvm@20.1.7"` pin and the builtins lookup + disappear from every iOS application. + +### 5.4 Criteria + +The combination this item creates, a prebuilt C library under a graph C++ +runtime, is testable on a Linux host with the same package, which is where +the engine's part is measured first: + +- Linux x86_64, `llvm@22.1.8`, a program that declares `llvm.libcxx` and + imports `std`: builds, `ldd` lists no `libc++`, the program runs; the + target-side report names `c++-abi libc++ (llvm.libcxx@22.1.8.1, graph)` + and `c-abi glibc (…, payload)`; the compile command carries `-nostdinc++` + and no payload `-isystem …/c++/v1`; the link command carries `-nostdlib++`. +- Negative direction, same host: the same program without the declaration + builds against the payload's libc++ exactly as today, byte for byte on + both command lines. Without this row the change could be read as "every + Linux build lost the payload's headers". +- On a macOS runner, `aarch64-ios-sim` with `llvm.libcxx` declared: a program + that imports `std`, uses `std::unordered_map` and + `std::atomic::wait`, links and prints under `simctl`; `otool -L` of + the artefact does not list `/usr/lib/libc++.1.dylib`; the link command + carries `-nostdlib++` and no `-lc++`. +- `aarch64-macos` with a floor: command lines unchanged from today, byte for + byte (the `SelfContained` row must not move). +- `aarch64-ios-sim` without the declaration: a program that does not import + `std` carries `-isystem /usr/include/c++/v1` and no payload + `-isystem`, links `-lc++` and prints no `target/cxx-runtime` line; one that + does still builds and the degradation names `llvm.libcxx`. +- Builtins: `aarch64-ios-sim` with `llvm.compiler-rt-builtins` declared and + a program whose source contains `if (__builtin_available(iOS 17, *))` + links and runs; without the declaration the same program fails at link + with `__isPlatformVersionAtLeast` undefined and prepare has printed the + degradation naming the package. Negative direction: `aarch64-macos` + prints no such degradation, since the payload carries `libclang_rt.osx.a`. +- The CI fixture's `toolchain = "llvm@20.1.7"` line, where one exists, is + deleted as part of the change, so the test measures the default payload. + +### 5.5 What lands where + +| piece | repository | size | +|---|---|---| +| `llvm.libcxx` 22.1.8.1: libc++ and libc++abi sources at `llvmorg-22.1.8`, generated configuration and module sources, manifest, examples, CI on Linux and on the iOS simulator | new repository `mcpplibs/libcxx` | one package | +| index entry, GitHub and GitCode release assets | `mcpp-index`, `mcpp-res` | one PR, one release | +| `cxxFromGraph` at the four sites, `-femulated-tls` narrowed, the older and current capability spellings, the SDK-header fallback, the builtins refusal, tests | mcpp | part of the batch PR | +| the Apple source selection (22.1.8.5) | `mcpplibs/compiler-rt-builtins` | one PR, one release, one index entry | +| `docs/20` iOS rows and `docs/22`; the 2026-09-11 record's table gains a superseded note pointing here | mcpp docs | text | + +The order between the repositories is the one every package-plus-engine +change has taken: the package is published and indexed first, the engine +PR's fixtures declare it (by `git` until the index carries it, by version +after), and the sandbox measures the released pair. + +## 6. Item 5: a known-key list that the parser does not read + +### 6.1 What the code does + +`toml.cppm:2635-2645` reads `min_api_level`; `2744-2747` defines +`kKnownTargetScalars = {cxx_runtime, linkage, sysroot, toolchain}` and +`kKnownTargetArrays = {runner}`; the sweep at `2748-2760` pushes "has +unsupported key '…' (ignored)" for every scalar not in the list and prints +the same list in the message. Tables are exempt from the sweep, which is why +`abi`, `runtime`, `build`, `xlings` and the rest are unaffected; +`min_api_level` is the only parsed non-table key that is missing. The +warning is `manifest/schema`, and `--strict` turns it into a failure +(`prepare.cppm:2058-2061`). e2e 641 asserts the value-range error and not the +absence of the schema warning, so it did not see the drift. + +### 6.2 Decision + +The one-line fix now: add `min_api_level` to `kKnownTargetScalars` and to +the message. The structural fix behind it, so that the list cannot drift +again (Rule D): the `[target.]` scalar parser becomes a table of +`{key, handler}` and the sweep reads the same table, in the shape the +`build.mcpp` directive table already has (`directives.cppm`). The table is +the smaller change once the one-liner is in, and it removes the "Supported +keys:" string as a third copy. + +### 6.3 Criteria + +- e2e 641 gains an assertion: a manifest with `min_api_level = 24` under + `mcpp build --strict --target ` exits 0 and its output + contains no `unsupported key`. +- Unit, with a denominator taken from the code: every `body.find("")` + in the `[target.]` parser names a key present in the table, and + every table key has a parse site. The denominator is read from the source + tree, not written into the test, so a fifth key added without a table row + fails the test rather than a build. + +## 7. Item 6: the tool store's key holds no source + +### 7.1 What the code does + +`tool_store::Key` (`tool_store.cppm:77-97`) is `{epoch, indexName, +packageName, version, targetName, hostTriple, compilerIdentity, profile, +features, upstreamKeys}`, the last being `name@version` strings of the +transitive closure; `entry_dir` is +`/tool//@//` (`178-183`). `prepare.cppm:8721-8745` +fills it from `[package] version` unconditionally; a store hit +(`entry_valid`, `8753`) skips the sub-build. `DepCacheIdentity` already +carries `sourceKind` (`4590-4599`) and the object cache uses it to keep +`path` and `git` packages out of the store (`11936-11938`), for the reason +`cache_key.cppm:44-49` states: they are "the only ones that can change +private flags without a version bump". The tool store applies no such rule. +The 2026-08-05 design for #355 specified the upstream axis as recursive +cache keys; what shipped was `name@version`, which for a `path` package is a +constant. + +### 7.2 Decision + +The source kind enters the decision, mirroring the object cache: + +| tool package source | key | store | +|---|---|---| +| index (`version`) | as today | as today: one version is immutable | +| `git` | `version` gains `+git.`, the commit the dependency resolved to (the identity the lock already computes) | stored: a commit is immutable content | +| `path` | `version` gains `+path.`, a hash over every regular file's relative path, size and modification time in sorted order, with `target/`, `.git/`, `.mcpp/` and the compile database excluded | stored: a stamp names one state of the tree, and an edit or its reversal each name another | + +The stamp is the answer to two constraints that pull apart. The criterion +needs both directions to move (an edit and its reversal each reach the +consumer), and e2e 187 asserts that a path tool is not rebuilt when nothing +changed. A store that never hits for `path` satisfies the first and fails the +second; a tree hash over file contents satisfies both at the cost of reading +every source on every build. The stat stamp satisfies both at one `stat` per +file, and it is what ninja itself trusts. Stale entries accumulate in the +store as the tree is edited, which `mcpp cache clean` already handles. + +`upstreamKeys` takes the same treatment per upstream, so that a `path` or +`git` package two levels below a tool moves the tool's key when it changes, +which is the case the comment on that field already describes. + +### 7.3 Criteria + +The 2026-09-08 measurement fixes the shape of the test: same path, different +bytes, both directions. With `examples/12`: edit the tool's emitter without +touching its version, build, and the consumer's output changes; revert the +edit, build, and it changes back. A version bump cannot be the probe, since +the tool's path is on the action's command line and the edge would rerun +regardless. For `git`: two commits of one tool repository, the root's `rev` +moved from one to the other and back, produce two store entries and the +consumer's output follows the pin. Negative direction: an index-sourced tool +builds once across two consuming projects (the store path is shared and the +"Building host tool" line appears once). + +## 8. Items 7 and A8: the descriptor's three blocks, and where a payload is declared + +### 8.1 Item 7: what the code does, and what the consumer actually experiences + +`emit_xpkg` fills `xpm..deps` from `manifest.xlings.workspaceByPlatform`, +the top-level table (`publisher.cppm:234-240`), and for every +`[target.''.xlings.workspace]` prints the `publish/target-axis-tools` +warning explaining that "a selector is not a platform" and that consumers +install what the three blocks name (`215-233`). The issue reports this +correctly and proposes deriving the blocks from the target axis and from +reexported host modules. + +Two facts change the consequence and the shape. First, a consumer's own +`mcpp build` loads each dependency's manifest, folds its target-axis tables +for the resolved target (`merge_conditional_config` at `7009-7012` calls +`merge_conditional_xlings`, `265-294`) and provisions the result through the +graph pass (`4541-4560`, `provision_xlings_addresses`). The GTK closure +declared on the rule package therefore reaches a consumer at build time +whether or not the descriptor names it; the descriptor decides what +`xlings install ` installs *before* any build, which matters for a +sandbox that installs and then builds offline, and for the auto-install gate +when it is closed. The failure mode is "installed later, or refused with the +package named", not "fails to link". This should be measured once in the +sandbox with auto-install off before the sentence enters `docs/`. + +Second, no reexport walk is needed at emit time: every mcpp package on the +index carries its own descriptor, and the consumer's mcpp resolves the rule +package as a package of its own, whose descriptor then names GTK once item 7 +is fixed for it. Deriving a library's blocks from its host modules would put +the same payload in two descriptors. + +### 8.2 Decision + +A selector that names only an operating system is a platform. `cfg(linux)`, +`cfg(os = "linux")`, `cfg(windows)`, `cfg(os = "windows")`, `cfg(macos)`, +`cfg(os = "macos")` and `cfg(unix)` (the first and third blocks) map onto the +descriptor's blocks; a predicate that mentions an architecture, an +environment, a layer or a feature keeps today's warning, whose text gains +the sentence "an OS-only selector is emitted; this one is not". The +`merge_conditional_xlings` rule for one package at two versions across the +axes (`xlings/axis-override`) applies to the emitted block as it does to the +build. + +### 8.3 Criteria + +- `mcpp emit xpkg` of a manifest with `[target.'cfg(linux)'.xlings.workspace]` + naming `xim:gtk4`: the `linux` block contains it, the other two do not, + and no warning is printed for that section. +- The same manifest with `cfg(target_arch = "aarch64")`: no block contains + the entry and the warning is printed, so the residual case is exercised. +- In the sandbox with auto-install off: install the published package, + build a consumer offline, and read whether the build refuses naming the + payload or links; the sentence in `docs/` is written from that reading. + +### 8.4 A8: declined again, and why the reason is a rule + +`fillXpkgDirs` (`5680-5751`) makes visible to a build program the owner's +own `[xlings.workspace]` and `[feature-xlings]` plus those of every host +module compiled into that build program; an ordinary library edge contributes +nothing. The 2026-09-12 record §2.8 read the issue's A8 as a declaration on +the wrong package and moved one line in HuxerUI. The present issue restates +the ask as "a reexported build-dependency's declarations reaching the +consumer would let an ordinary library declare its own payloads". + +The reason to decline is the same and is better stated as a rule: **a +payload is declared by the package whose code consumes it.** A build program +consumes a payload through `xpkg_dir`, so the declaration belongs to the +host module the build program is compiled from. A library consumes a payload +by compiling and linking against it, and declares it on its own target axis, +where the consumer's mcpp already folds and provisions it (§8.1). There is no +third consumer, so there is no third place. What the issue calls a workaround +(the GTK table living in `huxerui-build-rules-gtk`) is the designed form, and +`docs/31` already says so (lines 341-349), so nothing moves. + +## 9. Item A9: the universal APK is the library route applied to an app + +`build_and_pack_library` takes a list of triples, prepares and builds each, +and stages the legs into one tree (`library_pipeline.cppm:113-`); the program +route refuses a second `--target` with "packing one executable for several +triples would need several executables" (`cmd_publish.cppm:83-86`, +`124-129`). Both are right for what they name. On the Android rows an `app` +target's artifact is `lib.so` (2026-09-12 record, A3), which is the +library route's input. + +**Decision.** The route is chosen by the artifact's form, not by the target's +kind: an `app` whose resolved format is a shared object accepts several +triples and is staged as `lib//lib.so` per leg, the ABI derived +from the triple's architecture as the 2026-09-12 record already specifies +(`aarch64` to `arm64-v8a`, `x86_64` to `x86_64`). One stage, one dispatch, +one APK. A universal Mach-O is a different mechanism (one fat file produced +by `lipo`) and is not this item. + +**Criteria.** `mcpp pack --format apk --target aarch64-linux-android --target +x86_64-linux-android`: one APK whose listing contains both `lib/arm64-v8a/` +and `lib/x86_64/`; with one triple, the APK is byte-identical to today's. +Negative direction: two triples for an `app` on a row whose artifact is an +executable (`x86_64-linux`) remain refused with today's message. + +## 10. Order, and what the ecosystem side deletes + +The order is by the size of the change and by what each unblocks; the issue's +suggested order is kept where it does not conflict with a dependency between +items. + +| step | item | size | what HuxerUI or Lib-Live2D deletes when it lands | +|---|---|---|---| +| 1 | 5 | one line, one assertion; then the table | nothing; the warning stops | +| 2 | 1 + 2 | one `switch` at the resolve hit, one docs table | `git = "/local/path", branch = …` in every application developed beside the framework | +| 3 | 3a | a reorder in `pack::run`, one manifest field | nothing yet; `dist-apple` gains a resource destination in `mcpp:plugins` | +| 4 | 6 | source kind in the tool store | the version bump on every tool change | +| 5 | 4, recipe | `xim:llvm-apple-runtimes@` in `xim-pkgindex`, built on a macOS runner | nothing yet | +| 6 | 4, engine | row column, install through the pin's channel, archives and builtins by path, `appleFloor`, the Apple-cross `HostCoupled` branch removed, one refusal | `toolchain = "llvm@20.1.7"` in every iOS application; the `libclang_rt.iossim.a` lookup and its Xcode fallthrough in the build program | +| 7 | 3b | the Mach-O reader; bundling after a measurement | nothing yet; the `.app` becomes runnable with its dylibs | +| 8 | 7 | OS-only selectors map to blocks | nothing; the descriptor becomes complete for the rule package | +| 9 | A9 | route by artifact form | two builds and a hand-merged APK | +| - | A8 | declined | nothing; `docs/31` already carries the sentence | + +Each step keeps its own criterion, so that none of them is folded into a +neighbour and lost when the neighbour ships (the 2026-09-12 record's rule +7). The engine steps land in one mcpp PR, which is the repository's rule for +a batch; step 5 precedes step 6 across repositories because the engine PR +declares a package that must already be indexed. + +## 11. Self-review + +- **Statements about the present.** Every "the code does" clause names a + line at `b63dc4e5`. The first draft deferred two measurements (whether the + iOS SDK ships module sources; whether the payload ships the iOS builtins); + the runtime package of §5.3 makes both irrelevant to the default path, and + the first survives only as a question for route S, which is not built + here. One consequence is deferred to the sandbox: what a consumer + experiences when the descriptor under-states a payload (§8.1). +- **Where the issue was corrected.** Item 1's rule is queue order, not the + root (§2.1). Item 3a's refusal already reaches the dispatch, with an empty + tree (§3.1). Item 7's consequence is install-time, not link-time, and the + reexport walk is unnecessary (§8.1). A8 was decided in the 2026-09-12 + record and the reason is restated as a rule (§8.4). Item 4's premise is + confirmed and the 2026-09-11 record's contrary sentence is identified + (§5.1). +- **Where the issue's proposal was not taken.** `[patch]` (§2.2); using the + SDK's libc++ headers and pinning the payload to them (§5.2, routes S and + N, recorded and not built); deriving descriptor blocks from reexported + host modules (§8.1). +- **Negative directions.** Every criterion section has one, since each of + these checks can pass while measuring nothing: a warning emitted + unconditionally (§2.4), a tar that silently drops the closure (§3.3), a + reader that reads the wrong slice (§4.3), a refusal that never runs (§5.4), + a test whose denominator is written by hand (§6.3), a probe that reruns for + the wrong reason (§7.3), a residual warning that disappeared (§8.3), a + route that widened to executables (§9). + +## 12. Task list, dependencies and repositories + +The batch is one mcpp PR (the repository's rule for a batch of engine +changes), one new package repository, one index PR, one plugins PR, and +a release with both mirrors. Tasks are listed with what they depend on so +that none is left half done when its neighbour ships. + +| id | task | repository | depends on | criterion (§) | +|---|---|---|---|---| +| T1 | `min_api_level` in the known list and the message; e2e 641 asserts no schema warning under `--strict`; a unit test whose denominator is the parser's `body.find` sites | mcpp | - | §6.3 | +| T2 | OS-only selectors map to the descriptor's platform blocks; residual warning text; unit or e2e for both directions | mcpp | - | §8.3 | +| T3 | `ResolvedRecord.sourceRef`/`fromRoot`; the six-row switch at the resolve hit; `dependency/source-override`; the constraint check against a checkout's version; `docs/05` en and zh table; e2e fixture with local git repositories, six cases | mcpp | - | §2.4 | +| T4 | `pack::run` reorder: stage program and declared files before the closure; stage manifest `closure`; pipeline warning text; tar/dir keep refusing; e2e byte-identical ELF tree | mcpp | - | §3.3 | +| T5 | `needed_names` for Mach-O (thin and fat), `@rpath` resolution, Mach-O row of `is_system_lib`; unit tests with checked-in Mach-O fixtures; the macOS e2e | mcpp | T4 | §4.3 | +| T6 | tool store: `git` keyed by commit, `path` never a hit; `upstreamKeys` per source kind; e2e in both directions with `examples/12` | mcpp | - | §7.3 | +| T7 | `llvm.libcxx` 22.1.8.1: repository, sources at `llvmorg-22.1.8`, generated configuration and module sources, manifest, examples, CI (Linux with `llvm@22.1.8`; macOS runner for `aarch64-ios-sim`) | `mcpplibs/libcxx` (new) | - | §5.4 | +| T7b | `llvm.compiler-rt-builtins` 22.1.8.5: the Apple source selection under `cfg(os = "ios")` and `cfg(os = "macos")`; CI on a macOS runner | `mcpplibs/compiler-rt-builtins` | - | §5.4 | +| T8 | engine: `cxxFromGraph` at the four sites; `-femulated-tls` narrowed; both capability spellings; SDK-header fallback and the std-module diagnostic; the unsupplied compiler-runtime degradation; Linux e2e with T7 by `git`; iOS CI fixture declares T7 and T7b | mcpp | T7, T7b | §5.4 | +| T9 | route by artifact form: an `app` whose artifact is a shared object takes the library route's several triples; `lib//` staging; e2e on the Android rows | mcpp | - | §9 | +| T10 | docs: `docs/05` (T3), `docs/20` and `docs/22` iOS rows (T8), `docs/30` stage manifest field (T4), zh mirrors; the 2026-09-11 record's superseded note | mcpp | T3, T4, T8 | structure and parity checks | +| T11 | `mcpp-index`: `llvm.libcxx` entry and the `llvm.compiler-rt-builtins` 22.1.8.5 entry (GitHub and GitCode assets); `mcpp-res` releases | `mcpp-index`, `mcpp-res` | T7, T7b | index `latest` names them; sandbox install | +| T12 | `mcpp:plugins`: `dist-apple` places the staged tree's deployed files at the bundle's resource destination | `mcpp-plugins` | T4 released | the `.app` carries the deployed file | +| T13 | release mcpp; bump the workspace pin; GitCode assets by `gtc`; index bump PR | mcpp, `mcpp-index` | T1-T10 merged, CI green | `origin/main` HEAD run green; sandbox `mcpp --version` | +| T14 | sandbox verification with `xlings subos … --sandbox --cmd`, CN mirror configured for both tools: T3 warning, T6 rebuild, T8 Linux program, T2 descriptor, T1 silence | sandbox | T11, T13 | one ok/FAILED line per claim | +| T15 | this record's status and §13 "what landed and where it was measured" | mcpp docs | T14 | - | + +Parallel groups: {T1, T2}, {T3}, {T4 then T5}, {T6, T9}, {T7 then T8} can +proceed at once; T10 follows its inputs; T11 follows T7; T12 follows the +release; T13 to T15 are sequential. + +**Progress (2026-09-13, evening).** T1 to T8 and T10 are on the batch branch +(mcpp-community/mcpp#631). T7 is published (`mcpplibs/libcxx`, tag +22.1.8.1; GitCode mirror byte-identical). T7b is merged and tagged +(mcpplibs/compiler-rt-builtins#1, 22.1.8.5). T11 is merged and published +(mcpplibs/mcpp-index#408; a program resolving `llvm.libcxx = "22.1.8.1"` +from the index built and ran on Linux). T12 is prepared on a plugins +branch and waits for the release pin. T9, T13, T14 and T15 follow. + +## 13. What landed, and where each claim was measured + +| item | landed as | measured | +|---|---|---| +| 1, 2 | `prepare.cppm`: `ResolvedRecord.sourceRef`/`fromRoot`, the six-row decision at the resolve hit, `dependency/source-override`; `docs/05` en and zh | e2e 661, six cases, on Linux, macOS and Windows shards of #631 | +| 3a | `pack.cppm`: `stage_declared` before the closure, `finish_without_closure`, `closure_unavailable_outcome`; `stage_tree.cppm`: `ClosureStatus` in the manifest; `pipeline.cppm`: the tree is handed over with `closure = not-walked` | e2e 662 (Linux), e2e 666 (macOS: a Mach-O program reaches a dispatched format with the deployed file and `closure = not-walked`; `--format dir` still refuses), unit tests for the outcome function and the manifest | +| 3b | `binfmt.cppm`: `macho_needed` (thin and fat, both byte orders), `resolve_macho_names`, the Mach-O row of `is_system_lib`; `needed_names` dispatches to it. The closure step still reports `not-walked` for Mach-O; bundling waits for the `LC_RPATH` measurement | `test_pack_binfmt` over generated fixtures | +| 4 | `hostflags.cppm`/`flags.cppm`: `cxxFromGraph` and `appleSdkCxxHeaders`; `model.cppm`: `-femulated-tls` only when the C library is the graph's; `prepare.cppm`: both capability spellings, the `-isysroot` on the package std module's command, the `target/cxx-runtime` and `target/compiler-runtime` degradations, `payloadCompilerRuntimeAbsent`; `docs/20` en and zh | e2e 663 on Linux (glibc under `llvm.libcxx`, both directions); `ci-macos-ios` on #631: `aarch64-ios` and `aarch64-ios-sim` build a program that imports `std`, hashes strings, notifies an atomic and takes an availability check, over the two packages, with the report naming `c++-abi libc++ (libcxx@22.1.8.1, graph)` and `compiler-runtime compiler-rt (compiler-rt-builtins@22.1.8.5, graph)`; the first run without `-isysroot` on the std module's command stopped on `mbstate_t`, which is the measurement behind that line | +| 5 | `toml.cppm`: `min_api_level` in the known list and the message; `test_target_scalar_keys` with the parser's own `body.find` sites as the denominator | e2e 641 case 9 under `--strict` | +| 6 | `tool_store.cppm`: `tree_stamp`; `prepare.cppm`: `DepCacheIdentity.sourceRef`, `source_keyed_version` for the tool and its upstreams; `docs/30`, the examples/12 README and the CI probe restated | e2e 665 (both directions, a store hit when unchanged, `git` by commit), e2e 187 unchanged, `test_tool_store`; the examples job of #631, whose first run measured that the old probe edits the tree it later reads and had to build its probe compiler from a copy | +| 7 | `prepare_inputs.cppm`: `cfgpred::os_only_platforms`; `publisher.cppm`: OS-only selectors fill the platform blocks, the warning says so | `test_cfg_os_only_platform`, `test_xpkg_emit` | +| A9 | `route.cppm`: `accepts_several_targets`; `pipeline.cppm`: `build_extra_android_legs`; `pack.cppm`: `lib//` per leg; `triple.cppm`: `android_abi` | e2e 664 on the Android rows (two ABIs in one tree, one triple unchanged, executables refused) | +| A8 | declined; `docs/31` already states the rule | - | +| T7, T7b, T11 | `mcpplibs/libcxx` 22.1.8.1; `mcpplibs/compiler-rt-builtins` 22.1.8.5; index entries | GitHub and GitCode archives byte-identical; a Linux program resolving `llvm.libcxx = "22.1.8.1"` from the published index built and printed `1-2-3`; the builtins package's macOS CI reads six symbols out of the simulator archive | + +Three things the batch measured that the design did not foresee: + +- **The package std module lost the SDK.** The block that adopts a package's + std module rebuilds `stdModuleTargetFlags` and replaced the `-isysroot` + the toolchain resolution had put there; the precompile stopped on + `mbstate_t`. The SDK is now appended in that block (§5.3, the first + `ci-macos-ios` run). +- **An exclusion in a manifest's source list is global.** The builtins + package's first Apple revision re-listed five routines the package-wide + list excluded, and the archive did not carry them; the exclusions moved + into the five M-profile blocks (compiler-rt-builtins#2). +- **A fixture that stands in for the C++ layer answers for its headers.** + e2e 304's stub `mcpp:c++-abi` provider included `` and lost the + payload's libc++ under the new rule, on the one host whose toolchain is + clang; it now includes the C header. e2e 661 was corrected for the + fixture-path hygiene rule, and 666 for `import std` under clang. + +The design's two refusals were withdrawn on review before landing: a graph +that imports `std` on an Apple cross target without a package builds as it +did yesterday and is reported once (§5.3), and a payload without a builtins +archive is reported once rather than refused (§5.3, item 7). diff --git a/.agents/docs/README.md b/.agents/docs/README.md index 31b28d975..0824381d9 100644 --- a/.agents/docs/README.md +++ b/.agents/docs/README.md @@ -18,7 +18,7 @@ superseded_by: 2026-09-07-....md # when status is superseded --- ``` -283 records. +284 records. ## By subject @@ -58,6 +58,7 @@ Records that declare one. Everything else is listed by date below. ### triage +- [What a framework and its ecosystem library still hit in the engine: the ten items of #630, read against the code](2026-09-13-630-what-a-framework-still-hits-in-the-engine.md) — landed - [The engine gaps left open after the SDK batch](2026-09-12-engine-gaps-after-the-sdk-batch.md) — landed - [Six open issues: what each one actually is, and what would answer it](2026-09-11-six-open-issues-analysis.md) — active @@ -66,6 +67,7 @@ Records that declare one. Everything else is listed by date below. ### 2026-09 - [Four upstream asks from a UI framework: what each one is under mcpp's design, and the combined plan](2026-09-13-four-upstream-asks-from-a-ui-framework.md) — landed +- [What a framework and its ecosystem library still hit in the engine: the ten items of #630, read against the code](2026-09-13-630-what-a-framework-still-hits-in-the-engine.md) — landed - [The engine gaps left open after the SDK batch](2026-09-12-engine-gaps-after-the-sdk-batch.md) — landed - [A verified Web run that asked the host for node](2026-09-12-a-verified-web-run-that-asked-the-host-for-node.md) — landed - [Implementation plan: a UI framework on Android, iOS and Web (#622)](2026-09-12-622-implementation-plan.md) — landed diff --git a/.github/workflows/ci-linux.yml b/.github/workflows/ci-linux.yml index 90dd66d3d..05fb6e466 100644 --- a/.github/workflows/ci-linux.yml +++ b/.github/workflows/ci-linux.yml @@ -366,27 +366,36 @@ jobs: # `n.value * 2` keeps `0` at `0`: a probe that changed every literal # would turn `while (b != 0)` into a loop that divides by zero, and # this step would report a crash rather than an answer. - probe_on() { sed -i 's/return std::format("{}", n.value);/return std::format("{}", n.value * 2);/' ../toyc/src/compile.cppm; } - probe_off() { sed -i 's/return std::format("{}", n.value \* 2);/return std::format("{}", n.value);/' ../toyc/src/compile.cppm; } - install_toyc() { - ( cd ../toyc && "$MCPP" build >/dev/null ) - cp "$(find ../toyc/target -name toyc -type f -perm -u+x | head -1)" "$1" + # THE PROBE COMPILER IS BUILT FROM A COPY OF THE TREE. The store's + # key carries a stamp of the tool's tree (#630, item 6), so touching + # `../toyc` would move the key and the next run would rebuild the + # tool from the restored source -- the isolating change here is + # different bytes at the SAME path, which needs the tree untouched. + probe_tree=/tmp/toyc-probe + rm -rf "$probe_tree"; cp -r ../toyc "$probe_tree"; rm -rf "$probe_tree/target" + probe_on() { sed -i 's/return std::format("{}", n.value);/return std::format("{}", n.value * 2);/' "$1/src/compile.cppm"; } + probe_off() { sed -i 's/return std::format("{}", n.value \* 2);/return std::format("{}", n.value);/' "$1/src/compile.cppm"; } + install_toyc() { # $1 = the tree to build, $2 = the store path to overwrite + ( cd "$1" && "$MCPP" build >/dev/null ) + cp "$(find "$1/target" -name toyc -type f -perm -u+x | head -1)" "$2" } # `mcpp cache dir` prints a legacy-directory note on a second line. + # The newest entry, since a tree edited earlier in this job may have + # left another one. store="$("$MCPP" cache dir | head -1)/tool" - cached="$(find "$store" -path '*toyc@0.1.0*/bin/toyc' | head -1)" + cached="$(ls -t $(find "$store" -path '*toyc@0.1.0*/bin/toyc') 2>/dev/null | head -1)" [ -n "$cached" ] || { echo "FAIL: no toyc in the tool store under $store"; exit 1; } - probe_on; install_toyc "$cached"; probe_off + probe_on "$probe_tree"; install_toyc "$probe_tree" "$cached"; probe_off "$probe_tree" "$MCPP" run > /tmp/toy3.log 2>&1 || true grep -q 'answer() = 168' /tmp/toy3.log || { cat /tmp/toy3.log echo "FAIL: a changed compiler binary did not reach the artifact." echo " rules-toy must declare the compiler among the action's inputs." exit 1; } - install_toyc "$cached" + install_toyc "$probe_tree" "$cached" "$MCPP" run > /tmp/toy4.log 2>&1 || true grep -q 'answer() = 42' /tmp/toy4.log || { cat /tmp/toy4.log @@ -395,25 +404,34 @@ jobs: echo " rules-toy must declare the compiler among the action's inputs." exit 1; } - # THE STORE HOLDS NO SOURCE CONTENT, which the example's README and - # docs/30 both state. Editing the compiler's sources at the same - # version changes nothing, because nothing rebuilds the tool. - probe_on + # THE STORE'S KEY HOLDS THE SOURCE (#630, item 6), which the + # example's README and docs/30 both state: editing the compiler's + # sources at the same version rebuilds the tool, and so does the + # reversal. Both directions, because a key that only ever grew + # would pass the first and fail the second. + sleep 1; probe_on ../toyc "$MCPP" run > /tmp/toy5.log 2>&1 || true - grep -q 'answer() = 42' /tmp/toy5.log || { + grep -q 'answer() = 168' /tmp/toy5.log || { cat /tmp/toy5.log - echo "FAIL: the tool store now sees source content." - echo " The example's README and docs/30 state that it does not; update them." + echo "FAIL: editing the compiler's sources at the same version did not reach the artifact." + echo " The tool store's key must carry the path package's tree stamp." + exit 1; } + sleep 1; probe_off ../toyc + "$MCPP" run > /tmp/toy5b.log 2>&1 || true + grep -q 'answer() = 42' /tmp/toy5b.log || { + cat /tmp/toy5b.log + echo "FAIL: reverting the compiler's sources did not reach the artifact." exit 1; } - # ... and bumping the version is the way out both of them offer. + # ... and a version bump still rebuilds it, as it always did. + sleep 1; probe_on ../toyc sed -i 's/^version = "0.1.0"/version = "0.1.1"/' ../toyc/mcpp.toml "$MCPP" run > /tmp/toy6.log 2>&1 || true sed -i 's/^version = "0.1.1"/version = "0.1.0"/' ../toyc/mcpp.toml - probe_off + probe_off ../toyc grep -q 'answer() = 168' /tmp/toy6.log || { cat /tmp/toy6.log; echo "FAIL: bumping the tool version did not rebuild it"; exit 1; } - echo "ok: the compiler is a declared input, and the store is keyed on the version" + echo "ok: the compiler is a declared input, and the store is keyed on the tool's source" - name: "Graphics example: render offscreen on lavapipe and assert the pixels" run: | diff --git a/.github/workflows/ci-macos-ios.yml b/.github/workflows/ci-macos-ios.yml index bbef757b2..3547be489 100644 --- a/.github/workflows/ci-macos-ios.yml +++ b/.github/workflows/ci-macos-ios.yml @@ -194,6 +194,19 @@ jobs: [build] ios_deployment_target = "18.0" + # THE C++ STANDARD LIBRARY AND THE COMPILER RUNTIME ARE PACKAGES ON + # THESE ROWS (#630). The payload's static libc++ is a macOS object + # and its resource directory carries no iOS builtins archive, so the + # engine used to link the SDK's libc++ under the payload's newer + # headers -- which fails at link on the first inline path the older + # dylib does not export. `llvm.libcxx` brings headers, module and + # objects as one release; `llvm.compiler-rt-builtins` brings + # `__isPlatformVersionAtLeast`. Declared by git until the index + # carries them. + [target.'cfg(os = "ios")'.dependencies] + llvm.libcxx = { git = "https://github.com/mcpplibs/libcxx.git", tag = "22.1.8.1" } + llvm.compiler-rt-builtins = { git = "https://github.com/mcpplibs/compiler-rt-builtins.git", tag = "22.1.8.5" } + # THE RUNNER IS AN ARGV PREFIX AND THE SESSION BELONGS TO A # PACKAGE. `simctl-run` comes from `xim:apple-simulator-tools`; it # chooses a device, boots it if it is not booted, waits, spawns, and @@ -212,8 +225,18 @@ jobs: TOML cat > /tmp/iostest/src/main.cpp << 'CPP' import std; + // The two inline paths that failed at link under the payload's + // headers over the SDK's libc++ (`__hash_memory`, + // `__atomic_notify_all_global_table`), and an availability check, + // which is `__isPlatformVersionAtLeast` from the builtins package. int main() { - std::vector v{3, 1, 2}; + std::unordered_map m; + m["three"] = 3; + std::atomic a{1}; + a.notify_all(); + int two = 2; + if (__builtin_available(iOS 17, *)) two = 2; + std::vector v{m["three"], a.load(), two}; std::ranges::sort(v); std::print("{}-{}-{}\n", v[0], v[1], v[2]); } @@ -274,9 +297,61 @@ jobs: run: | set -euo pipefail cd /tmp/iostest - "$MCPP_DEV" build --target aarch64-ios-sim + "$MCPP_DEV" build --target aarch64-ios-sim 2>&1 | tee build-sim.log /tmp/assert-artefact.sh aarch64-ios-sim arm64 7 18.0 + # THE TWO LAYERS THE PACKAGES SUPPLY, READ FROM THE REPORT AND FROM THE + # ARTEFACT. A build that succeeds cannot tell a self-contained libc++ + # from the SDK's; the load commands can, and the report says which + # package answered for each layer. + - name: "aarch64-ios-sim: the C++ runtime and the builtins are the graph's" + run: | + set -euo pipefail + cd /tmp/iostest + grep -E 'c\+\+-abi +libc\+\+ +\(libcxx@22\.1\.8\.1, graph\)' build-sim.log \ + || { echo "FAIL: the report does not name llvm.libcxx as the C++ layer"; exit 1; } + grep -E 'compiler-runtime +compiler-rt +\(compiler-rt-builtins@22\.1\.8\.5, graph\)' build-sim.log \ + || { echo "FAIL: the report does not name llvm.compiler-rt-builtins as the compiler runtime"; exit 1; } + art=$(ls /tmp/iostest/target/aarch64-ios-sim/*/bin/iostest | head -1) + if otool -L "$art" | grep -q 'libc++'; then + echo "FAIL: the artefact links a libc++ dylib"; otool -L "$art"; exit 1 + fi + echo "ok: no libc++ dylib in the load commands; both layers are the graph's" + + # THE NEGATIVE DIRECTION: without the packages a program that imports + # std still builds, as it did before this release, and the degradation + # names the two lines; one that does not import std takes the SDK's + # headers under the SDK's libc++. Without this step the change could be + # read as "every iOS build now needs a package". + - name: "aarch64-ios-sim: without the packages, import std builds with the hazard named and plain C++ takes the SDK's headers" + run: | + set -euo pipefail + rm -rf /tmp/iosplain && mkdir -p /tmp/iosplain/src && cd /tmp/iosplain + cat > mcpp.toml << 'TOML' + [package] + name = "iosplain" + version = "0.1.0" + [build] + ios_deployment_target = "18.0" + TOML + printf 'import std;\nint main() { std::print("x\\n"); }\n' > src/main.cpp + "$MCPP_DEV" build --target aarch64-ios-sim > mixed.log 2>&1 || { echo "FAIL: import std without llvm.libcxx no longer builds"; cat mixed.log; exit 1; } + # A degradation renders its `what` text, not its domain, so the + # sentence is what a log can be asked for. + grep -q "links the SDK's libc++ under the toolchain payload's libc++ headers" mixed.log || { echo "FAIL: no degradation named the mixed libc++"; cat mixed.log; exit 1; } + grep -q 'llvm.libcxx' mixed.log || { echo "FAIL: the degradation does not name llvm.libcxx"; cat mixed.log; exit 1; } + printf '#include \n#include \nint main() { std::string s = "1-2-3"; std::puts(s.c_str()); }\n' > src/main.cpp + rm -rf target + "$MCPP_DEV" build --target aarch64-ios-sim 2>&1 | tee plain.log + grep -q 'carries no compiler runtime for aarch64-ios-sim' plain.log || { echo "FAIL: no degradation named the missing compiler runtime"; exit 1; } + grep -q "links the SDK's libc++ under" plain.log && { echo "FAIL: a program without import std was reported as mixing libc++"; exit 1; } + ninja=$(ls target/aarch64-ios-sim/*/build.ninja | head -1) + grep -q -- '-isystem[^ ]*iPhoneSimulator[^ ]*/usr/include/c++/v1' "$ninja" || { echo "FAIL: the plain program does not take the SDK's C++ headers"; grep -o -- '-isystem[^ ]*c++/v1' "$ninja" | sort -u; exit 1; } + grep -q -- '-isystem[^ ]*xim-x-llvm[^ ]*/c++/v1' "$ninja" && { echo "FAIL: the plain program still takes the payload's C++ headers"; exit 1; } + art=$(ls target/aarch64-ios-sim/*/bin/iosplain | head -1) + otool -L "$art" | grep -q '/usr/lib/libc++.1.dylib' || { echo "FAIL: the plain program does not link the SDK's libc++"; otool -L "$art"; exit 1; } + echo "ok: import std builds with the hazard named; plain C++ takes the SDK's headers and libc++; the builtins degradation is printed" + # THE SUPPORTED PATH, which is what the `verified` tier claims: a runner # the manifest declares and a program a package provides. The program's # own line is compared whole. An iOS-simulator Mach-O does not execute on diff --git a/CHANGELOG.md b/CHANGELOG.md index c96783c5f..0b9e6c88f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,68 @@ ## [Unreleased] +### 一个框架与它的生态库仍会撞到的引擎缺口:#630 的十项 + +#630 汇总了 HuxerUI 在六个平台上落地后引擎仍欠的十项。每一项都先在 +`b63dc4e5` 上定位到代码行再分类,其中四处与 issue 的叙述不同(设计记录 §0): +「根赢」其实是工作队列先出队者赢;`mcpp pack` 早已把 Mach-O 的拒绝带到 +dispatched 格式,只是交出去的树是空的;第 7 项的后果在安装期而不在链接期; +A8 在 2026-09-12 的记录里已判定为声明放错了包。 + +**一个身份,两条声明,要说出来(项 1、2)。** 同一依赖身份的两条 `git`/`path` +声明此前没有任何字段被比较,留下的是先出队的那条。现在根的声明跨 kind、跨引用 +都赢,并以 `dependency/source-override` 告警点名双方与各自引用;依赖对根所钉 +checkout 的版本要求照 `addrset::unify` 的 Holds/Violated 校验;两个非根声明的 +kind 冲突仍拒绝,并提示在根里声明以决之。`docs/05` 中英文各得一张决策表。 + +**先暂存声明的文件,再走发现的闭包(项 3a)。** `mcpp pack` 在闭包之前拷贝程序 +与 `mcpp::deploy` 的文件;本机走不了闭包的格式(Mach-O 程序;Windows 宿主上的 +非 PE 产物)把树交给 dispatched 格式,stage manifest 记 `closure = not-walked` +与原因;`tar`/`dir` 仍拒绝。`binfmt` 得到 Mach-O 读取器(thin 与 fat、两种字节 +序、`LC_LOAD_DYLIB`/`LC_RPATH`)、`is_system_lib` 的 Mach-O 行与 `@rpath` 解析 +器(项 3b);闭包步骤对 Mach-O 仍报 not-walked,打包 dylib 待 `LC_RPATH` 改写 +的实测。 + +**C++ 层回答自己的头文件(项 4)。** 四处用 `cAbi.prebuilt()` 代答「载荷的 +libc++ 是否适用」,在 openkal 与原生构建重合,在 iOS 行(SDK 的 C 库 + 图里的 +libc++)分开:引擎把 libc++ 22 的头配到 SDK 的 libc++ 19 上,程序在 +`__hash_memory` 处链接失败。现改读 `plan.targetSide.cxx.fromGraph()`;iOS 行的 +C++ 运行时与编译器运行时成为图里的源码包(`llvm.libcxx@22.1.8.1`、 +`llvm.compiler-rt-builtins@22.1.8.5`,与 `openkal-llvm-runtime` 同一机制,框架 +声明一次、应用继承)。不声明时:不导入 `std` 的程序取 SDK 的头;导入的保留昨天 +的搭配并由 prepare 报告一次(`target/cxx-runtime`)点名两行;载荷没有该平台的 +builtins 归档时报告一次(`target/compiler-runtime`),从不去 Xcode 里找。 +`graph_runtime_compile_flags` 只在 C 库来自图时给 Mach-O 加 `-femulated-tls`。 + +**`min_api_level` 是已知键(项 5)。** 它被 Android 行读取、被 `dist-apk` 写成 +`minSdkVersion`,却被 `[target.]` 的未知键扫描报成 unsupported,`--strict` +下变硬错。加入已知表;单测的分母取自解析器自己的 `body.find` 站点。 + +**工具仓库的键里有源码(项 6)。** host 工具按 包×版本×宿主×编译器×feature×闭包 +版本 键控,`path` 包改源码不抬版本即被无视。现在 `git` 包按解析出的 commit、 +`path` 包按树的 stat 印记(相对路径、大小、mtime)进键,上游同理;改动与回退各 +到达消费者,未变的树仍命中(e2e 187 的断言不变)。 + +**只点操作系统的 selector 就是平台(项 7)。** `mcpp emit xpkg` 把 `cfg(linux)`、 +`cfg(os = "windows")`、`cfg(macos)`、`cfg(unix)` 映射到描述符的平台块;其余 +selector 保留告警并说明这一例外。 + +**一个 app 的产物是共享库时,pack 接受多个 triple(A9)。** 路由按产物形态而非 +target 的 kind:Android 行的 `kind = "app"` 走库路线的多腿暂存,一棵树里 +`lib//lib.so` 各一份,一次 dispatch 得到 universal APK。 + +- 判据:`tests/e2e/661`(六种声明组合,双向)、`662`(先暂存后闭包,dispatched + 格式看到部署文件)、`663`(Linux 上以 glibc 为 C 库、`llvm.libcxx` 为 C++ 层: + 报告、`-nostdinc++`、`-nostdlib++`、`ldd` 无 libc++、程序运行;不带包时命令行 + 逐字节不变)、`665`(工具跟随源码,双向,未变命中)、`666`(macOS:Mach-O 程序 + 被暂存并交给 dispatched 格式,`closure = not-walked`)、`641` 第 9 例(`--strict` + 下 `min_api_level` 静默);`ci-macos-ios` 在模拟器行上用两个包构建并运行,并断言 + 不带包的两个方向;单测 `test_hostflags`、`test_targetside`、`test_tool_store`、 + `test_pack_binfmt`、`test_target_scalar_keys`、`test_cfg_os_only_platform`。 +- 生态:`mcpplibs/libcxx`(新仓)、`mcpplibs/compiler-rt-builtins#1`、 + `mcpplibs/mcpp-index#408`,GitHub 与 GitCode 双端资产逐字节核验。 +- 设计记录:`.agents/docs/2026-09-13-630-what-a-framework-still-hits-in-the-engine.md`。 + ### `mcpp::action` 的列表不再有长度上限;`${mcpp.self}` 让 action 叫出引擎自己 内置的 `mcpp` 模块此前把一个 action 的 `inputs`、`outputs`、`command` 等六个列表 diff --git a/docs/05-dependencies.md b/docs/05-dependencies.md index cc40a69da..f5ff8dd91 100644 --- a/docs/05-dependencies.md +++ b/docs/05-dependencies.md @@ -112,6 +112,34 @@ baz = "=1.2.3" # Exact match qux = ">=1.0, <2.0" # Range combination ``` +### When two declarations of one dependency disagree + +Two edges in the dependency graph can name the same identity — the same +`(namespace, name)` pair — from two different places: the root manifest and a +dependency's own `[dependencies]`, or two unrelated dependencies. A declaration +has a **kind** (`version`, `git`, or `path`) and a **reference** within that +kind (a SemVer constraint, a `git` URL plus `rev`/`tag`/`branch`, or a +filesystem path). mcpp resolves the identity once; every requester's edge is +recorded, and what one requester's declaration decides is what every other +requester of the same identity gets. + +| first declaration | second declaration | outcome | +|---|---|---| +| any | same kind, same reference | Unchanged: the second declaration becomes an edge to the identity already resolved. Nothing is reported. | +| `version` | `version`, a different constraint | The two constraints are AND-combined by SemVer, as above; an unsatisfiable pair is refused, naming both constraints and both requesters. | +| `git` | `git`, a different `rev`/`tag`/`branch` | The root's declaration wins when the root is one of the two requesters; otherwise the declaration resolved first wins. A `dependency/source-override` warning names both requesters and both references, states which one is used and why, and how to take the other. This is never silent. | +| `path` | `path`, a different directory | The same rule and the same warning as the row above, compared by canonical absolute directory rather than by git reference. | +| a root `path` or `git` declaration | a dependency's `git` or `version` declaration (a kind clash) | The root's declaration wins, with the same warning. When the losing declaration is a `version` requirement, it is checked against the `[package] version` of the root's resolved checkout; a violated requirement is refused, naming the pin, the requester and the requirement. | +| a dependency's `path`/`git` declaration | another dependency's declaration of a different kind (a kind clash, and neither party is the root) | Refused: "requested as both a … dep … and a … dep …. Pick one." The message adds one sentence: declare the identity in the root to settle it. | + +The root's privilege here is bounded the same way `linkage` is bounded to the +root manifest's own edges (see `[dependencies]` above): a whole-graph choice of +*which checkout* an identity resolves to is a decision only the artifact's own +manifest may make silently on a dependency's behalf. A dependency that +disagrees with another dependency, with neither being the root, is never +settled by guessing which one was declared first — that is exactly the +"accident of queue order" this section replaces. + ### Namespace resolution rules Every package has a two-part identity: a **namespace** and a **name**. Every diff --git a/docs/10-pack-and-release.md b/docs/10-pack-and-release.md index 172e22b04..38d6f95f3 100644 --- a/docs/10-pack-and-release.md +++ b/docs/10-pack-and-release.md @@ -191,6 +191,20 @@ on every row, whatever file that row links it to (see prebuilt binaries per triple and has no single staged tree, so `mcpp pack --format ` is refused rather than ignored. +**A `kind = "app"` target whose artifact is a shared object accepts more than +one `--target`** (mcpp 2026.9.13.2+): on every Android row an application +*is* the shared library the platform loads, so `mcpp pack myapp --target +aarch64-linux-android --target x86_64-linux-android` builds and stages both +legs into one tree, exactly as a library package's several triples already +do. Each leg lands at `lib//lib.so` (`aarch64` → `arm64-v8a`, +`x86_64` → `x86_64`), the declared deploy files are staged once, and one +dispatch runs against the combined tree — which is what lets a member such as +`dist-apk` build one universal APK. A single `--target` keeps today's flat +`lib/lib.so` layout unchanged. A target whose artifact is an executable +on any requested row is still refused for a second `--target`: packing one +executable for several triples would need several executables, which is a +different mechanism (`lipo`'s universal binary) that this does not provide. + When `-o` is given a bare filename, the output is placed under `target/dist/`; when it includes a directory (relative or absolute), the literal path is used. @@ -482,6 +496,19 @@ Linux either. A `kind = "lib"` / `"shared"` target packs normally on macOS — a library package never runs the artifact. This restriction is only for programs. +The dependency closure can now be **read** rather than run — `mcpp.pack.binfmt` +walks a Mach-O's load commands (`LC_LOAD_DYLIB` and its weak/re-export/upward +siblings for names, `LC_RPATH` for search entries) the same way it already +reads a PE's import table, and resolves `@executable_path`, `@loader_path` and +`@rpath` the way `dyld` would, without loading anything. `mcpp pack` does not +call it yet: bundling a resolved dylib beside the program needs an editor for +`LC_RPATH` (a load command has no free space to grow into), and that editor is +designed once resolution is measured on a real macOS build, not before. Until +then a Mach-O program still stages without its closure — see [Producing a +distributable](30-build-mcpp.md#producing-a-distributable-pack_format--stage_dir-20269111) +for what a dispatched format (`.app`, `.ipa`) can already do with a tree that +has a program and no closure. + ## Configuration Packaging behavior is configured via the `[pack]` section in `mcpp.toml`. The @@ -513,11 +540,12 @@ The `static` mode additionally requires a musl toolchain configured under ## Planned Support -macOS **program** bundling (the Mach-O dependency closure, via `otool -L` / -`LC_LOAD_DYLIB`, and `install_name_tool` for relocation) is still on the -roadmap; until it lands `mcpp pack ` refuses on that format rather than -producing something that only looks like a bundle. Windows DLL bundling beyond -the current `.zip` is also on the roadmap. +macOS **program** bundling is still on the roadmap. The closure is read now +(`mcpp.pack.binfmt`'s Mach-O load-command walk — no `otool` needed), but +bundling a resolved dylib beside the program and rewriting `LC_RPATH` for it +is not; until that lands `mcpp pack ` still refuses on that format +rather than producing something that only looks like a bundle. Windows DLL +bundling beyond the current `.zip` is also on the roadmap. Distribution formats such as `.deb`, `.rpm`, AppImage and `.msi` are **not** on this list, and that is a decision rather than an omission: they live in diff --git a/docs/20-toolchains.md b/docs/20-toolchains.md index e57939cbc..e1382f1aa 100644 --- a/docs/20-toolchains.md +++ b/docs/20-toolchains.md @@ -671,6 +671,36 @@ something a dependency can supply, so there is nothing a later step could learn that would change the answer -- and a machine without Xcode should not download a compiler before being told the compiler is not what is missing. +### The C++ runtime and the compiler runtime are packages on these rows + +The payload's static libc++ is a macOS object, which ld64 refuses in an iOS +link, and its resource directory carries `libclang_rt.osx.a` and no `ios` or +`iossim` archive. Both layers therefore come from the dependency graph, as +the C library and the builtins do on the bare-metal rows: + +```toml +[target.'cfg(os = "ios")'.dependencies] +llvm.libcxx = "22.1.8.1" # libc++ and libc++abi as source, with the std module +llvm.compiler-rt-builtins = "22.1.8.5" # __isPlatformVersionAtLeast and the generic routines +``` + +A framework declares the two lines once and every application inherits them. +The report names both layers as the graph's, the link carries `-nostdlib++`, +and the artifact's load commands name no `libc++.1.dylib`: the headers a +translation unit is compiled against, the module it imports and the objects it +links are one release by construction. + +Without the first declaration the runtime is the SDK's libc++. A program that +does not import `std` then takes the SDK's headers too +(`-nostdinc++ -isystem /usr/include/c++/v1`) and links `-lc++`. A program +that imports `std` keeps the payload's module and headers over the SDK's +dylib, as every iOS build did before this release; the two are different +releases of libc++, and prepare reports the pairing once, naming the two lines +above, because it links until an inline path in the newer headers names an +export the older dylib lacks. Without the second declaration, prepare reports +once that the payload has no compiler runtime for the platform; a program that +never reaches an availability check links regardless. + ### The deployment target `[build] ios_deployment_target` sits beside `macos_deployment_target`, and the diff --git a/docs/22-target-side.md b/docs/22-target-side.md index b9083426f..da2a93b50 100644 --- a/docs/22-target-side.md +++ b/docs/22-target-side.md @@ -370,7 +370,7 @@ The selector `` has three forms: |---|---|---| | **bare OS alias** | a single OS / family — the concise, common form | `[target.windows]`, `[target.unix]` | | **`cfg(...)` predicate** | a compound condition (arch / env / combinators) | `[target.'cfg(all(linux, not(arch = "aarch64")))']` | -| **exact triple** | one specific target (also carries `toolchain` / `linkage` / `sysroot` / `runner`; see [04 §2.7.3](04-mcpp-toml.md)) | `[target.x86_64-linux-musl]` | +| **exact triple** | one specific target (also carries `toolchain` / `linkage` / `sysroot` / `runner` / `min_api_level`; see [04 §2.7.3](04-mcpp-toml.md)) | `[target.x86_64-linux-musl]` | A selector may carry platform-conditional **dependencies** and **build flags**: diff --git a/docs/23-the-project-environment.md b/docs/23-the-project-environment.md index fe70bcd56..19ea23d2f 100644 --- a/docs/23-the-project-environment.md +++ b/docs/23-the-project-environment.md @@ -334,12 +334,15 @@ this applies to. `subos` is not conditional on a target: a project has one environment, so `[target..xlings]` refuses the key rather than dropping it. -**A published descriptor carries no edge for a target-axis entry**, and -`mcpp publish` says so. A descriptor has one block per platform, and a selector -is not a platform — `cfg(target_arch = "aarch64")` names no block that file has. -What a CONSUMER of the package gets installed comes from the top-level -`[xlings.workspace]`; the target axis stays correct for what the package's own -build compiles against. +**A published descriptor carries no edge for a target-axis entry in general**, +and `mcpp publish` says so. A descriptor has one block per platform, and most +selectors are not a platform — `cfg(target_arch = "aarch64")` names no block +that file has. **A selector that names only an operating system IS a +platform**, though: `cfg(linux)`, `cfg(os = "linux")` and the windows/macos/unix +equivalents fold into the matching `xpm..deps` block(s) instead of +only raising the advisory. What a CONSUMER of the package gets installed +otherwise comes from the top-level `[xlings.workspace]`; the target axis stays +correct for what the package's own build compiles against. See [SPEC-004](specs/manifest-semantics.md) for the general rule these two axes are an instance of. diff --git a/docs/30-build-mcpp.md b/docs/30-build-mcpp.md index 35db532b7..5360e8506 100644 --- a/docs/30-build-mcpp.md +++ b/docs/30-build-mcpp.md @@ -709,6 +709,17 @@ changes, and a closure that grew a dependency's shared library while the program's own bytes did not would leave the previous distributable in place, reported as up to date. +**The manifest's first line is `closure = walked` or `closure = not-walked`.** +`mcpp pack` stages the program and its declared runtime +files before it asks whether this host can resolve the artifact's dependency +closure, so the tree can exist without one — a Mach-O program today, or a +non-PE artifact packed from a Windows host. `--format tar` and `--format dir` +still fail the command in that case, since the archive IS the closure; a +dispatched format receives the tree regardless, with a second manifest line, +`reason = `, naming the mechanism that was unavailable. A provider that +needs the closure reads the field rather than inferring a gap from an empty +`lib/`. + Commands are an **argv, not a shell string** (no shell is assumed — Windows has none to rely on), and the only interpolations are a closed set: @@ -1110,23 +1121,28 @@ Four properties worth knowing: consumer's to pay. A package gates the expensive part with `[features]` + `required_features` (protobuf's `protoc` needs libprotoc's ~157 extra TUs, which the runtime's users must not compile). -- **Cached globally**, keyed on package version × host toolchain × features × - its own dependency closure — built once per machine, not once per project. - -**The key holds no source content, and for a `path` dependency that is visible.** -A published version is immutable, so for a tool that arrives from an index the -key is exact. A tool being edited next door has the same version from one build -to the next, and the cached binary stays: measured on -[`examples/12-a-new-device-language`](../examples/12-a-new-device-language/), -a change to the tool's emitter left `mcpp run` printing the previous answer, -while bumping the tool package's version rebuilt it and changed the artifact. -Bump the version, or empty the build cache with `mcpp cache clean` — the tool -store lives inside it, at `/tool//@/`. - -This is a gap in the rebuild, not in the tracking. An action that declares the -tool among its inputs does re-run when that file's bytes change, measured by -overwriting the binary in the store: the artifact followed. What does not happen -is the rebuild that would change those bytes. +- **Cached globally**, keyed on the package's source × host toolchain × + features × its own dependency closure — built once per machine, not once + per project. + +**The key holds the source, in the form each source kind can offer.** A +published version is immutable, so for a tool that arrives from an index the +version alone identifies its sources. A `git` tool is keyed by the commit it +resolved to. A `path` tool has no version that moves when its sources do, so +it is keyed by a stamp of its tree: every regular file's relative path, size +and modification time, with `target/`, `.git/`, `.mcpp/` and the compile +database excluded. An edit to the tool's emitter therefore reaches the +consumer on the next build, and so does its reversal, while a tree that did +not change is a store hit and is not rebuilt. Measured on +[`examples/12-a-new-device-language`](../examples/12-a-new-device-language/); +before this rule the key held the version alone and a change to the emitter +left `mcpp run` printing the previous answer until the version was bumped. +Entries accumulate as a tree is edited; `mcpp cache clean` empties the store, +which lives at `/tool//@[+]/`. + +The action's own tracking is separate from the store's key. An action that +declares the tool among its inputs re-runs when that file's bytes change, +measured by overwriting the binary in the store: the artifact followed. ### `[tools.overrides]` — use an existing binary diff --git a/docs/zh/05-dependencies.md b/docs/zh/05-dependencies.md index 1909c6dc0..e02141945 100644 --- a/docs/zh/05-dependencies.md +++ b/docs/zh/05-dependencies.md @@ -103,6 +103,30 @@ baz = "=1.2.3" # 精确匹配 qux = ">=1.0, <2.0" # 范围组合 ``` +### 同一依赖两条声明冲突的处理 + +依赖图里的两条边可能指向同一个身份 —— 同一个 `(namespace, name)` 二元组 —— +却来自图中两个不同的位置:根 manifest 与某个依赖自己的 `[dependencies]`,或者 +两个互不相干的依赖。一条声明有一个**种类**(`version`、`git` 或 `path`)和该种 +类下的一个**引用**(一条 SemVer 约束、一个 `git` URL 加 `rev`/`tag`/`branch`, +或一个文件系统路径)。mcpp 只解析这个身份一次;每个请求方的边都会被记录,而 +其中一个请求方的声明决定了同一身份下所有其他请求方拿到的是什么。 + +| 第一条声明 | 第二条声明 | 结果 | +|---|---|---| +| 任意 | 同种类、同引用 | 不变:第二条声明成为指向已解析身份的一条边,不报告任何信息。 | +| `version` | `version`,不同的约束 | 按上文所述用 SemVer 对两个约束做 AND 合并;无法满足的一对被拒绝,并点名两个约束与两个请求方。 | +| `git` | `git`,不同的 `rev`/`tag`/`branch` | 若根是两个请求方之一,根的声明胜出;否则先解析出来的声明胜出。`dependency/source-override` 警告点名两个请求方与两个引用,说明哪一个被采用、原因是什么,以及如何取用另一个。这个警告绝不会被吞掉。 | +| `path` | `path`,不同的目录 | 规则与警告同上一行,只是比较的是规范化后的绝对目录而不是 git 引用。 | +| 根的 `path` 或 `git` 声明 | 某个依赖的 `git` 或 `version` 声明(种类冲突) | 根的声明胜出,警告同上。若败下阵的声明是一条 `version` 需求,它会被拿去与根已解析出的那份 checkout 的 `[package] version` 核对;需求被违反就拒绝,并点名那个 pin、请求方与需求本身。 | +| 某个依赖的 `path`/`git` 声明 | 另一个依赖的不同种类的声明(种类冲突,且双方都不是根) | 拒绝:"requested as both a … dep … and a … dep …. Pick one."。消息多出一句:在根里声明该身份即可解决。 | + +这里根所拥有的特权,与 `linkage` 只在根 manifest 自己的边上生效(见上文 +`[dependencies]`)是同一条边界:一个身份最终解析到*哪一份 checkout*,是一个 +只有构件自己的 manifest 才能替某个依赖悄悄做出的整图级决定。两个依赖互相冲 +突、且都不是根的情形,绝不会靠猜哪个先被声明来解决 —— 那正是本节要替换掉的 +"队列顺序的意外"。 + ### 命名空间解析规则 每个包的身份是**命名空间 + 名字**二元组。每个 selector 都只规范化成一个身份: diff --git a/docs/zh/10-pack-and-release.md b/docs/zh/10-pack-and-release.md index 8b825bd76..b5a3b3755 100644 --- a/docs/zh/10-pack-and-release.md +++ b/docs/zh/10-pack-and-release.md @@ -152,6 +152,17 @@ error: unknown --format 'bogus'. 的预构建产物,没有单独一棵暂存树,所以 `mcpp pack <库> --format ` 会被拒绝, 而不是被忽略。 +**产物是共享目标文件的 `kind = "app"` target 可以接受一个以上的 `--target`** +(mcpp 2026.9.13.2+):在每一行 Android 上,一个应用*就是*平台加载的那个共享库, +所以 `mcpp pack myapp --target aarch64-linux-android --target x86_64-linux-android` +会构建并把两条腿暂存进同一棵树里,与库包的多三元组做法完全一致。每条腿落在 +`lib//lib.so`(`aarch64` → `arm64-v8a`,`x86_64` → `x86_64`),声明的 +部署文件只暂存一次,随后对这棵合并后的树只跑一次分派——这正是 `dist-apk` 这样的 +成员能构建出一个通用 APK 的原因。只给一个 `--target` 时,今天这种扁平的 +`lib/lib.so` 布局保持不变。产物在任何被请求的一行上是可执行文件的 target, +第二个 `--target` 依旧被拒绝:为多个三元组打包一个可执行文件需要多个可执行文件, +那是另一种机制(`lipo` 的通用二进制),不在此列。 + `-o` 接受裸文件名时自动归到 `target/dist/`;含目录(相对或绝对) 时按字面路径输出。 @@ -408,6 +419,17 @@ mcpp pack --target x86_64-windows-gnu # 在 Linux 宿主上 `kind = "lib"` / `"shared"` 目标在 macOS 上照常打包 —— 库打包从不运行产物。 这条限制只针对程序。 +依赖闭包现在可以被**读出来**而不必运行了 —— `mcpp.pack.binfmt` 走一遍 Mach-O 的 +load command(`LC_LOAD_DYLIB` 及其 weak/re-export/upward 三个变体给出名字, +`LC_RPATH` 给出搜索项),读法与它读 PE 导入表一样;并按 `dyld` 的规则解析 +`@executable_path`、`@loader_path`、`@rpath`,全程不加载任何东西。`mcpp pack` +还没有调用它:把解析到的 dylib 拷到程序旁边、再重写它的 `LC_RPATH`,需要一个 +load command 编辑器(一条 load command 里没有空间可以塞进更长的路径),这个编辑器 +要等在真实的 macOS 构建上量过解析结果之后才设计,而不是先设计。在那之前,Mach-O +程序仍然是"暂存但没有闭包"——一个被分发出去的格式(`.app`、`.ipa`)已经能拿这样一棵 +"有程序、没闭包"的树做什么,见 +[产出可分发物](30-build-mcpp.md#产出可分发物pack_format-与-stage_dir20269111)。 + ## 配置项 打包行为通过 `mcpp.toml` 中的 `[pack]` 节配置,常用字段如下: @@ -435,10 +457,11 @@ force_bundle = ["libfoo.so"] # 即使命中 PEP 600 名单也强制打包 ## 待支持 -macOS **程序** bundling(Mach-O 依赖闭包,走 `otool -L` / `LC_LOAD_DYLIB`, -重定位走 `install_name_tool`)仍在规划中;在它落地之前,`mcpp pack <程序>` -会在该格式上拒绝,而不是产出一个只是看起来像 bundle 的东西。当前 `.zip` -之外的 Windows DLL 分发,同样在规划中。 +macOS **程序** bundling 仍在规划中。闭包现在已经能读出来了(`mcpp.pack.binfmt` +走一遍 Mach-O 的 load command,不需要 `otool`),但把解析到的 dylib 拷到程序旁边、 +再重写它的 `LC_RPATH` 还没做;在这落地之前,`mcpp pack <程序>` 仍会在该格式上拒绝, +而不是产出一个只是看起来像 bundle 的东西。当前 `.zip` 之外的 Windows DLL 分发, +同样在规划中。 `.deb`、`.rpm`、AppImage、`.msi` 这些分发格式**不在**这份清单上,而这是一个决定而不是 一处遗漏:它们住在包里,经 `--format ` 到达用户,理由见上一节。`[pack]` 的内建 diff --git a/docs/zh/20-toolchains.md b/docs/zh/20-toolchains.md index e59fcb8db..19e99d7e2 100644 --- a/docs/zh/20-toolchains.md +++ b/docs/zh/20-toolchains.md @@ -606,6 +606,29 @@ error: target aarch64-ios needs the iphoneos SDK, which this machine does not pr 没有任何后续步骤能学到会改变这个答案的信息 —— 而一台没有 Xcode 的机器不应该先下载 一个编译器,然后才被告知缺的不是编译器。 +### 这些行上的 C++ 运行时与编译器运行时是包 + +载荷的静态 libc++ 是 macOS 的目标文件,ld64 拒绝把它链进 iOS 的链接;载荷的资源目录里只有 +`libclang_rt.osx.a`,没有 `ios` 或 `iossim` 的归档。因此这两层都来自依赖图,与裸机行上的 +C 库和 builtins 同一做法: + +```toml +[target.'cfg(os = "ios")'.dependencies] +llvm.libcxx = "22.1.8.1" # libc++ 与 libc++abi 的源码,带 std 模块 +llvm.compiler-rt-builtins = "22.1.8.5" # __isPlatformVersionAtLeast 与通用例程 +``` + +框架声明一次,每个应用通过依赖边继承。报告把两层都记为图里的,链接行带 `-nostdlib++`, +产物的加载命令不含 `libc++.1.dylib`:翻译单元编译所用的头、导入的模块与链接的目标文件按构造 +是同一个发布版本。 + +不声明第一行时,运行时是 SDK 的 libc++。不导入 `std` 的程序随之取 SDK 的头文件 +(`-nostdinc++ -isystem /usr/include/c++/v1`)并链接 `-lc++`。导入 `std` 的程序仍用载荷的 +模块与头文件配 SDK 的 dylib,与此版本之前每一次 iOS 构建相同;二者是 libc++ 的两个发布版本, +prepare 报告一次这对搭配并点名上面两行,因为它只在较新头文件里的内联路径没有引用旧 dylib 缺失的 +导出时才能链接。不声明第二行时,prepare 报告一次「载荷没有这个平台的编译器运行时」;从不触及 +可用性检查的程序照常链接。 + ### 部署目标 `[build] ios_deployment_target` 与 `macos_deployment_target` 并列,而两者是两个键 diff --git a/docs/zh/22-target-side.md b/docs/zh/22-target-side.md index 6843ac006..435ef1ce7 100644 --- a/docs/zh/22-target-side.md +++ b/docs/zh/22-target-side.md @@ -312,7 +312,7 @@ mcpp 不认识的键 —— 打错的字,或来自更新版本 mcpp 的谓词 |---|---|---| | **裸 OS 别名** | 单个 OS / 族 —— 简洁且常用的形式 | `[target.windows]`、`[target.unix]` | | **`cfg(...)` 谓词** | 复合条件(arch / env / 组合子) | `[target.'cfg(all(linux, not(arch = "aarch64")))']` | -| **精确三元组** | 某个具体目标(同时承载 `toolchain` / `linkage` / `sysroot` / `runner`,见 [04 §2.7.3](04-mcpp-toml.md)) | `[target.x86_64-linux-musl]` | +| **精确三元组** | 某个具体目标(同时承载 `toolchain` / `linkage` / `sysroot` / `runner` / `min_api_level`,见 [04 §2.7.3](04-mcpp-toml.md)) | `[target.x86_64-linux-musl]` | 一个选择器可以承载平台条件的**依赖**与**构建 flag**: diff --git a/docs/zh/23-the-project-environment.md b/docs/zh/23-the-project-environment.md index 264e2b85b..813aa10f8 100644 --- a/docs/zh/23-the-project-environment.md +++ b/docs/zh/23-the-project-environment.md @@ -275,10 +275,12 @@ this applies to. `subos` 不按目标条件化:一个工程只有一个环境,所以 `[target..xlings]` 拒绝这个键,而不是把它丢掉。 -**已发布的描述符不为目标轴条目携带边**,`mcpp publish` 会说明这一点。描述符按平台 -分块,而 selector 不是平台 —— `cfg(target_arch = "aarch64")` 不对应那份文件里的任何一块。 -**使用者**装到的东西来自顶层 `[xlings.workspace]`;目标轴对"本包自己的构建对着什么" -仍然是正确的。 +**已发布的描述符一般不为目标轴条目携带边**,`mcpp publish` 会说明这一点。描述符按 +平台分块,大多数 selector 不是平台 —— `cfg(target_arch = "aarch64")` 不对应那份文件 +里的任何一块。**但只命名一个操作系统的 selector 就是平台**:`cfg(linux)`、 +`cfg(os = "linux")` 以及 windows/macos/unix 的对应写法,会被并入相应的 +`xpm..deps` 块,而不只是给出提示。除此之外,**使用者**装到的东西来自顶层 +`[xlings.workspace]`;目标轴对"本包自己的构建对着什么"仍然是正确的。 这两条轴所属的一般规则见 [SPEC-004](../specs/manifest-semantics.md)。 diff --git a/docs/zh/30-build-mcpp.md b/docs/zh/30-build-mcpp.md index 062d4d57f..5f496aa81 100644 --- a/docs/zh/30-build-mcpp.md +++ b/docs/zh/30-build-mcpp.md @@ -606,6 +606,14 @@ mcpp 会写出 `<暂存树>.stage-manifest` —— 一个兄弟文件,永不是 依赖的共享库、同时程序自己的字节并没有变的情况,会把上一次的可分发物原地留下,并报告为 已是最新。 +**这份 manifest 的第一行是 `closure = walked` 或 `closure = not-walked`。** +`mcpp pack` 会先暂存程序本身与它声明过的运行期文件,再去问这台宿主机能不能解析该产物的 +依赖闭包 —— 所以这棵树可以在没有闭包的情况下存在:今天是一个 Mach-O 程序,或者在 +Windows 宿主上打包一个非 PE 产物。`--format tar` 与 `--format dir` 在这种情况下仍然让 +命令失败,因为归档本身就是闭包;一个被分发出去的格式无论如何都会拿到这棵树,manifest 上 +多出第二行 `reason = <原因>`,点名是哪种机制在这台宿主上不可用。需要闭包的提供方读这个 +字段,而不是从一个空的 `lib/` 里去猜测缺口。 + 命令是 **argv 而不是 shell 字符串**(不假设存在 shell —— Windows 没有能依赖的那个), 插值只有封闭的一组: @@ -937,20 +945,21 @@ grpc = { version = "1.83.0", tools = ["grpc_cpp_plugin"] } - **默认关闭。** 没人要就什么都不构建,成本由消费者付。包用 `[features]` + `required_features` 给昂贵的部分加门(protobuf 的 `protoc` 需要 libprotoc 的 ~157 个额外 TU,只用运行时的人绝不该编译它)。 -- **全局缓存**,按 包版本 × host 工具链 × feature × 自身依赖闭包 键控 —— 每台机器 +- **全局缓存**,按 包的源码 × host 工具链 × feature × 自身依赖闭包 键控 —— 每台机器 构建一次,而不是每个工程一次。 -**这个键里没有源码内容,而对 `path` 依赖这一点是看得见的。** 已发布的版本不可变, -所以对来自索引的工具,这个键是精确的。而正在旁边被编辑的工具,两次构建之间版本相同, -缓存里的二进制就留在原地:在 +**这个键里有源码,以每种来源能给出的形式。** 已发布的版本不可变,所以对来自索引的 +工具,版本本身就标识了源码。`git` 工具按解析出的 commit 键控。`path` 工具没有随源码 +移动的版本,所以按树的印记键控:每个普通文件的相对路径、大小与修改时间,不计 +`target/`、`.git/`、`.mcpp/` 与 compile database。于是改动工具的 emitter 在下一次构建 +到达消费者,改回去也到达,而没有变化的树仍是 store 命中、不重建。在 [`examples/12-a-new-device-language`](../../examples/12-a-new-device-language/) -上实测,改动工具的 emitter 之后 `mcpp run` 打印的是上一次的答案,而抬高工具包的版本 -之后它被重建、产物随之改变。抬版本,或用 `mcpp cache clean` 清空构建缓存 —— -tool store 就住在里面,路径是 `/tool//@/`。 +上实测;此前这个键只有版本,改动 emitter 之后 `mcpp run` 打印的是上一次的答案,直到 +版本被抬高。树被编辑时条目会累积;`mcpp cache clean` 清空 store,路径是 +`/tool//@[+]/`。 -**缺口在重建,不在跟踪。** 把工具列进 action 输入的规则,确实会在那个文件的字节变化 -时重跑 —— 实测直接覆盖 store 里的二进制,产物随之改变。不发生的是「让这些字节变化」 -的那次重建。 +action 自己的跟踪与 store 的键是两回事。把工具列进 action 输入的规则,会在那个文件的 +字节变化时重跑 —— 实测直接覆盖 store 里的二进制,产物随之改变。 ### `[tools.overrides]` —— 使用已有的二进制 diff --git a/examples/12-a-new-device-language/README.md b/examples/12-a-new-device-language/README.md index bcaba4862..d6998d1c0 100644 --- a/examples/12-a-new-device-language/README.md +++ b/examples/12-a-new-device-language/README.md @@ -11,7 +11,7 @@ mcpp run ``` Rules example.rules.toy (example:rules-toy) - Building host tool toyc:toyc from toyc v0.1.0 (once per package version × host toolchain) + Building host tool toyc:toyc from toyc v0.1.0 (once per package source and host toolchain) Compiling toyapp v0.1.0 (.) Finished dev [unoptimized + debuginfo] in 0.65s @@ -182,30 +182,33 @@ a real compiler rather than a script. | editing the `.toy` reaches the artifact | `scale(…, 2)` → `scale(…, 3)`: `42` → `63` | | the compiler is built for the build machine, on demand, and only when the rule is active | the `Building host tool` line above; no store entry without the feature | -## The boundary this example measured: a host tool is cached by version +## The boundary this example measured: a host tool is cached by its source Four changes were made one at a time, each from the same starting state: | what changed | the artifact | how it was changed | |---|---|---| | the `.toy` source | follows: `42` → `63` | `scale(…, 2)` → `scale(…, 3)` | -| the compiler's **bytes**, at the path the action names | follows: `42` → `168` | overwriting the binary in the tool store | -| the compiler's **sources**, its version unchanged | does not follow: the previous answer stands | editing the emitter | +| the compiler's **bytes**, at the path the action names | follows: `42` → `168` | overwriting the binary in the tool store, from a copy of the tree | +| the compiler's **sources**, its version unchanged | follows: `42` → `168`, and the tool is rebuilt | editing the emitter | | the compiler's **version** | follows: `42` → `168`, and the tool is rebuilt | `0.1.0` → `0.1.1` | -Rows two and three are the whole finding, and they separate two things that are -easy to merge. **The action's input tracking works**: `rules-toy` declares the -compiler beside the source, and changing that file's bytes re-runs the edge. -**What does not happen is the rebuild that would change those bytes.** The tool -store's key is the tool package's identity, version, host triple, compiler -identity, profile, features and the versions of its transitive dependencies — -it holds no source content. For a package that arrives from an index the key is -exact, because a published version is immutable; for a `path` dependency being -edited it is not. - -`mcpp run` prints `Finished dev in 0.00s` in row three, and that line is mcpp's -own summary rather than evidence: row two prints it too, and the artifact -changed. +Rows two and three separate two things that are easy to merge. **The action's +input tracking** is one: `rules-toy` declares the compiler beside the source, +and changing that file's bytes re-runs the edge. **The store's key** is the +other: it holds the tool package's identity, version, host triple, compiler +identity, profile, features, the versions of its transitive dependencies, and +the tool's source in the form its kind offers. For a package that arrives from +an index the version alone identifies the sources, because a published version +is immutable. For a `path` dependency being edited the key carries a stamp of +the tree (every file's relative path, size and modification time), so row +three rebuilds the tool and the artifact follows. Row three used to read "does +not follow: the previous answer stands", and that reading was the measurement +mcpp#630 (item 6) removed. + +`mcpp run` prints `Finished dev in 0.00s` when nothing changed, and that line +is mcpp's own summary rather than evidence: row two prints it too, and the +artifact changed. **Row four does not test row two, which is why the difference is worth stating.** The tool's path is on the action's command line, so a new version re-runs the @@ -216,9 +219,9 @@ row two — and it takes both of its directions: with the input removed, the artifact followed the overwrite and then stopped following the restore. CI runs that pair. -Two ways out: bump the tool package's version, or empty the build cache with -`mcpp cache clean` — the tool store lives inside it, at -`/tool//@/`. +Entries accumulate in the store as a tree is edited, one per stamp; +`mcpp cache clean` empties it. The store lives at +`/tool//@[+]/`. **One more trap sits behind them.** Going back from `0.1.1` to `0.1.0`, whose clean tool was still in the store, left the artifact at `168`. The build program diff --git a/mcpp.toml b/mcpp.toml index bf2ddfa98..6b9508bce 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.9.13.1" +version = "2026.9.13.2" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/modules/buildmcpp/src/tool_store.cppm b/modules/buildmcpp/src/tool_store.cppm index 5f15b4bad..000045875 100644 --- a/modules/buildmcpp/src/tool_store.cppm +++ b/modules/buildmcpp/src/tool_store.cppm @@ -99,6 +99,27 @@ struct Key { std::string key_hex(const Key& k); nlohmann::json to_json(const Key& k); +// THE IDENTITY OF A SOURCE TREE THAT HAS NO VERSION OF ITS OWN. +// +// An index package's `name@version` names immutable content, so the version +// alone identifies the bytes a tool was built from. A `path` package changes +// under an unchanged version, and a `git` package at a branch moves; the store +// keyed on the version alone then keeps a binary built from sources that no +// longer exist. Measured (2026-09-08, examples/12): an emitter change in a path +// tool package reached the consumer only after the package's version was +// bumped, and until then every build reported success over the previous +// compiler's output. +// +// A `git` package is keyed by its resolved commit, which is immutable content. +// A `path` package is keyed by this stamp: every regular file's relative path, +// size and modification time, in sorted order, hashed. It is what ninja itself +// trusts to decide a rebuild, costs one `stat` per file rather than a read, +// and moves in both directions -- an edit and its reversal each produce a new +// key, which is what the criterion demands. Build products, the version +// control directory and the engine's own scratch are excluded, since they +// change without the sources changing. +std::string tree_stamp(const std::filesystem::path& root); + // /tool//@// std::filesystem::path entry_dir(const std::filesystem::path& cacheRoot, const Key& k); std::filesystem::path bin_path(const std::filesystem::path& entryDir, @@ -171,6 +192,31 @@ nlohmann::json to_json(const Key& k) { return j; } +std::string tree_stamp(const fs::path& root) { + std::vector rows; + std::error_code ec; + fs::recursive_directory_iterator it(root, fs::directory_options::skip_permission_denied, ec); + for (; it != fs::recursive_directory_iterator(); it.increment(ec)) { + if (ec) break; + const auto& p = it->path(); + const auto name = p.filename().string(); + if (it->is_directory(ec)) { + if (name == "target" || name == ".git" || name == ".mcpp") it.disable_recursion_pending(); + continue; + } + if (!it->is_regular_file(ec)) continue; + if (name == "compile_commands.json") continue; + const auto rel = p.lexically_relative(root).generic_string(); + const auto sz = fs::file_size(p, ec); + const auto mt = fs::last_write_time(p, ec).time_since_epoch().count(); + rows.push_back(std::format("{}|{}|{}", rel, sz, mt)); + } + std::ranges::sort(rows); + std::string joined; + for (auto const& r : rows) { joined += r; joined += '\n'; } + return mcpp::toolchain::hash_string(joined); +} + std::string key_hex(const Key& k) { return mcpp::toolchain::hash_string(to_json(k).dump()); } diff --git a/modules/manifest/src/targetside_model.cppm b/modules/manifest/src/targetside_model.cppm index e1c9ac1e6..6f5f7b35a 100644 --- a/modules/manifest/src/targetside_model.cppm +++ b/modules/manifest/src/targetside_model.cppm @@ -411,6 +411,13 @@ struct Inputs { EnvAxis envAxis = EnvAxis::Unknown; std::optional compilerRuntime; + // THE PAYLOAD SHIPS NO COMPILER RUNTIME FOR THIS TARGET. Set by the caller + // when it has looked: clang's official macOS payload carries + // `libclang_rt.osx.a` and no `ios`/`iossim` archive (measured, 22.1.8), + // and the Darwin driver links nothing rather than failing when the file + // is absent. With no graph provider the layer is then genuinely absent, + // and saying so is what lets the report and a diagnostic name it. + bool payloadCompilerRuntimeAbsent = false; std::optional kernelAbi; std::optional cAbi; std::optional cxxAbi; @@ -512,6 +519,8 @@ inline TargetSide resolve(const Inputs& in) { ts.compilerRuntime = { Origin::Graph, in.compilerRuntime->display_interface(), in.compilerRuntime->id(), false }; + else if (in.payloadCompilerRuntimeAbsent) + ts.compilerRuntime = { Origin::None, {}, {}, false }; else if (!in.compilerFamily.empty()) ts.compilerRuntime = { Origin::Payload, in.compilerFamily, {}, false }; diff --git a/modules/manifest/src/toml.cppm b/modules/manifest/src/toml.cppm index c47ad9914..7e7611248 100644 --- a/modules/manifest/src/toml.cppm +++ b/modules/manifest/src/toml.cppm @@ -2741,8 +2741,15 @@ std::expected parse_string(std::string_view content, // correctly spelled array is not; the message prints both in one // alphabetical line, because a reader of the warning should not // have to know a key's type to find it there. + // #610 / #630 item 5 — `min_api_level` was parsed above but + // missing from this list, so a correctly spelled key was reported + // as unsupported and, under `--strict`, turned into a hard error. + // tests/unit/test_manifest.cpp reads the parse sites above by + // source text and checks both directions against these two + // arrays, so a sixth key added to the parser without a matching + // entry here fails that test instead of shipping silently again. static constexpr std::string_view kKnownTargetScalars[] = { - "cxx_runtime", "linkage", "sysroot", "toolchain", + "cxx_runtime", "linkage", "min_api_level", "sysroot", "toolchain", }; static constexpr std::string_view kKnownTargetArrays[] = { "runner" }; for (auto& [key, value] : body) { @@ -2753,8 +2760,8 @@ std::expected parse_string(std::string_view content, if (std::ranges::find(known, key) != known.end()) continue; m.schemaWarnings.push_back(std::format( "[target.{}] has unsupported key '{}' (ignored). Supported keys: " - "cxx_runtime, linkage, runner, sysroot, toolchain, plus the " - "[target..runners] table for named runners. " + "cxx_runtime, linkage, min_api_level, runner, sysroot, toolchain, " + "plus the [target..runners] table for named runners. " "Per-role contracts go in [build].cxx_runtime's table form.", triple, key)); } diff --git a/modules/toolchain-model/src/model.cppm b/modules/toolchain-model/src/model.cppm index 34e25ee65..a717f7bf1 100644 --- a/modules/toolchain-model/src/model.cppm +++ b/modules/toolchain-model/src/model.cppm @@ -205,6 +205,15 @@ struct Toolchain { // so the SDK has to travel explicitly, on the compile side as well as the // link side. std::filesystem::path appleSdkRoot; + // THE C++ HEADERS ARE THE SDK'S, on an Apple cross target whose C++ + // runtime is the SDK's libc++ and whose graph does not import `std`. + // Decided by prepare, read by the compile-flag builder: with no graph + // package the runtime is the SDK's by construction, and the headers + // follow the runtime. When the graph imports `std` the payload's module + // and headers stay in use over the SDK's dylib, which is the pairing + // that fails at link on the first inline path the older dylib lacks, and + // prepare reports that once rather than refusing what built yesterday. + bool appleSdkCxxHeaders = false; std::vector compilerRuntimeDirs; // LD_LIBRARY_PATH for private tools std::vector linkRuntimeDirs; // -L/-rpath dirs for produced binaries // Environment the toolchain's tools need when invoked (set on the ninja @@ -525,7 +534,14 @@ std::vector graph_runtime_compile_flags(const Toolchain& tc) { // `os == "macos"` used to miss. Both need the emulated-TLS model for the // same reason PE does -- `_tlv_bootstrap` is loader-bootstrapped there // exactly as `_tls_index` is on PE. - if (t->is_pe() || t->is_mach_o()) out.emplace_back("-femulated-tls"); + // + // AND ONLY WHILE THE C LIBRARY IS THE GRAPH'S. `_tlv_bootstrap` is + // supplied by dyld, and openkal-macos has no dyld; a hosted Mach-O target + // whose C library is the SDK's (the iOS rows over `llvm.libcxx`) has the + // loader and the native TLS model with it. Emulated TLS there would be a + // graph-wide ABI choice made for a reason that does not apply. + if (t->is_pe() || (t->is_mach_o() && !tc.cAbiPrebuilt)) + out.emplace_back("-femulated-tls"); // MACH-O ONLY, AND THE REASON IS THAT WEAK-DEF IS A RUN-TIME MECHANISM // THERE. See the note on this function for the measurement. `is_mach_o()` // rather than `os == "macos"`: the mechanism is ld64's, which iOS shares. diff --git a/modules/toolchain-model/src/triple.cppm b/modules/toolchain-model/src/triple.cppm index a5ca71065..d08c889e0 100644 --- a/modules/toolchain-model/src/triple.cppm +++ b/modules/toolchain-model/src/triple.cppm @@ -1183,6 +1183,22 @@ inline ApplicationForm application_form(const Triple& t) { : ApplicationForm::Executable; } +// THE ANDROID ABI NAME OF A ROW'S ARCHITECTURE (#630 A9), for the +// `lib//lib.so` layout every Android packaging tool uses +// (`aapt2`'s `lib/` convention, and mcpp's own multi-target staging when a +// `kind = "app"` target whose form is `SharedObject` is packed for more than +// one triple at once). Android coined its own vocabulary for the +// architectures it supports rather than reusing GNU's; the two rows mcpp +// currently produces a shared object for (`aarch64`, `x86_64`) are the only +// ones this translates, and an architecture Android has no name for keeps +// its GNU spelling — mcpp has never had reason to invent a fourth +// architecture vocabulary, and guessing one here would be exactly that. +inline std::string android_abi(const Triple& t) { + if (t.arch == "aarch64") return "arm64-v8a"; + if (t.arch == "x86_64") return "x86_64"; + return t.arch; +} + // THE PLATFORM NAME OF A ROW, for `[package] platforms` (#622 A7). A platform // name is the triple's `os`, except where an `env` names a platform of its // own: Android is `linux` with `env = "android"` in the triple, and a diff --git a/modules/versioning/src/version.cppm b/modules/versioning/src/version.cppm index 9ac984c33..dd503e982 100644 --- a/modules/versioning/src/version.cppm +++ b/modules/versioning/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.9.13.1"; +inline constexpr std::string_view MCPP_VERSION = "2026.9.13.2"; } // namespace mcpp diff --git a/src/build/distribution.cppm b/src/build/distribution.cppm index 4e7c7a156..c7377cbf8 100644 --- a/src/build/distribution.cppm +++ b/src/build/distribution.cppm @@ -495,15 +495,19 @@ Mechanism resolve(const MechanismInput& in) { } return m; } - // iOS TAKES ITS C++ RUNTIME FROM THE SDK, AND HAS NO OTHER OPTION. + // iOS TAKES ITS C++ RUNTIME FROM THE SDK UNLESS THE GRAPH SUPPLIES ONE. // // Every iOS release ships libc++ in the OS, and the SDK's // `libc++.tbd` is the stub that links against it -- so `-lc++` is - // both the correct and the only answer for these rows. The two - // alternatives are closed by construction rather than by policy: the - // payload's static archives are built for macOS and ld64 refuses - // them in an iOS link, and the payload's libc++.dylib is not present - // on a device at all. + // the answer the PAYLOAD can give for these rows. The two payload + // alternatives are closed by construction rather than by policy: its + // static archives are built for macOS and ld64 refuses them in an + // iOS link, and its libc++.dylib is not present on a device at all. + // A graph package (`llvm.libcxx`, libc++ as source) is the other + // option, and it is decided before this switch: `graphCxxRuntime` + // returns `SelfContained` with `-nostdlib++` and this branch is not + // reached (mcpp#630). What remains here is the no-package case, whose + // headers `hostflags.cppm` takes from the SDK for the same reason. // // The contract vocabulary calls this `HostCoupled`, which reads // oddly for a cross target; what it means in every cell is "the C++ diff --git a/src/build/flags.cppm b/src/build/flags.cppm index a64da785f..acf1b2be3 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -614,6 +614,11 @@ CompileFlags compute_flags(const BuildPlan& plan) { // is what makes it impossible for them to disagree — which they did, // from #511 until now, because only the link side was corrected. hopt.cAbiPrebuilt = plan.targetSide.cAbi.prebuilt(); + // AND THE C++ LAYER'S OWN ANSWER, which the C library's used to stand + // in for. The two differ on a hosted target whose C library is a + // located SDK while a package supplies libc++ (mcpp#630, §5). + hopt.cxxFromGraph = plan.targetSide.cxx.fromGraph(); + hopt.appleSdkCxxHeaders = plan.toolchain.appleSdkCxxHeaders; compile_toolchain_flags = mcpp::toolchain::render_tokens( mcpp::toolchain::host_compile_tokens(plan.toolchain, hopt, ninjaEsc)); } else { @@ -1157,13 +1162,16 @@ CompileFlags compute_flags(const BuildPlan& plan) { // // measured on `openkal-linux = "0.5.4"` with a `throw` in main. // - // THE C LIBRARY IS WHAT DECIDES IT, for the same reason it decides - // the link line's search paths: the payload's C++ runtime was - // configured against the payload's C library, so it is eligible when - // and only when that C library is the one in use. `check_layering` - // states the same rule in the other direction, refusing the - // combination this predicate must not create. - mi.graphCxxRuntime = !plan.targetSide.cAbi.prebuilt(); + // THE C++ LAYER DECIDES IT. This read `!cAbi.prebuilt()` while the + // only graph C++ runtime sat over a graph C library, and the two + // questions were one; `check_layering` still refuses the payload's + // libc++ over a graph C library, so a graph C library implies a graph + // C++ runtime. The converse does not hold: a hosted target whose C + // library is a located SDK can take libc++ from a package + // (`llvm.libcxx` on the iOS rows, mcpp#630), and asking the C library + // there answered "payload" and linked the SDK's `-lc++` under objects + // compiled against the package's headers. + mi.graphCxxRuntime = plan.targetSide.cxx.fromGraph(); const bool wantsArchives = (base == dist::Contract::SelfContained diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 9e7dca679..3f0dad8f2 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -4596,6 +4596,11 @@ prepare_build(bool print_fingerprint, // version-keyed directory, so name@version identifies its sources. // Path and git checkouts can change under an unchanged identity. std::string sourceKind; + // What identifies the SOURCES when the version does not: the resolved + // commit for `git`, the package root for `path`, empty for an index + // package. Read by the tool store, whose key must hold everything + // that can change a built tool's bytes (#630, item 6). + std::string sourceRef; }; std::vector dep_cache_identities; struct GitLockIdentity { @@ -4614,6 +4619,24 @@ prepare_build(bool print_fingerprint, std::string constraint; // AND-combined original constraints (version src only) std::string requestedBy; // human-readable for error messages std::string source; // "version" | "path" | "git" — for type-clash check + // The declaration's identity beyond `source`, so a SECOND declaration + // of the same (ns, name) can be compared for "the same reference" + // rather than merely "the same kind". `git`: "#=", + // from the DECLARED ref (never the resolved commit — comparing two + // branch names must not need a network round trip to decide whether + // they conflict). `path`: the canonical absolute directory. `version`: + // the original constraint string ("*" for none). See the + // `dependency/source-override` decision at the resolve hit (2026-09-13 + // #630 record, §2.2). + std::string sourceRef; + // True when this record's declaration came from the root manifest's + // own [dependencies]/[dev-dependencies]/[build-dependencies] + // (`item.consumerDepIndex == kMainConsumer` at the time the record + // was created). Bounds the root's privilege to override a + // conflicting declaration of the SAME identity the way + // `DependencySpec::linkage` is honoured only on the root's own + // edges — see dep_spec.cppm. + bool fromRoot = false; // Reached ONLY through [dev-dependencies]. mcpp.lock excludes these: // dev-deps are resolved under `mcpp test` and not under `mcpp build`, so // recording them makes a VCS-committed file depend on which command ran @@ -6431,6 +6454,31 @@ prepare_build(bool print_fingerprint, /*buildOnly=*/true}); } + // `ResolvedRecord::sourceRef` for a given declaration — see the field's + // comment. Computed from what was AUTHORED, not from a network round + // trip: a `branch` reference is compared by name here, and the two + // clones it may eventually resolve to are a question `resolveSemver`-style + // ANSWERING code, not this IDENTITY code, would have to ask. + auto sourceRefOf = [&](const std::string& kind, + const mcpp::manifest::DependencySpec& s, + const std::filesystem::path& resolveRoot, + const std::string& originalConstraint) -> std::string { + if (kind == "git") { + return std::format("{}#{}={}", s.git, s.gitRefKind, s.gitRev); + } + if (kind == "path") { + std::filesystem::path p = s.path; + auto base = resolveRoot.empty() ? *root : resolveRoot; + if (p.is_relative()) p = base / p; + std::error_code ec; + auto canon = std::filesystem::weakly_canonical(p, ec); + return (ec ? p : canon).lexically_normal().generic_string(); + } + // "version": the constraint as authored; empty means unconstrained, + // matching `addrset::unify`'s treatment of a bare-name claim. + return originalConstraint.empty() ? std::string("*") : originalConstraint; + }; + while (!worklist.empty()) { auto item = std::move(worklist.front()); worklist.pop_front(); @@ -6471,19 +6519,114 @@ prepare_build(bool print_fingerprint, spec.isPath() ? "path" : spec.isGit() ? "git" : "version"; + // The commit a `git` dependency resolved to, carried out of the clone + // branch below for the cache identity. + std::string sourceCommit; if (auto it = resolved.find(key); it != resolved.end()) { // A package is dev-only until some non-dev consumer wants it. Order // of arrival must not decide, so this is an AND over every request. it->second.devOnly = it->second.devOnly && item.devOnly; - // Conflict detection. + // Conflict detection: a KIND clash (`path`/`git`/`version` differ). + // Rows 4 and 5 of the decision table in the 2026-09-13-630 record + // §2.2. Two non-root requesters keep the outright refusal (row + // 5); when the root is a party, its declaration wins instead + // (row 4) — a whole-graph choice of WHICH checkout an identity + // resolves to is exactly the kind of decision + // `DependencySpec::linkage` already reserves to the root's own + // edges (dep_spec.cppm). if (it->second.source != sourceKind) { - return std::unexpected(std::format( - "dependency '{}{}{}' is requested as both a {} dep " - "(by '{}') and a {} dep (by '{}'). Pick one.", + const bool existingIsRoot = it->second.fromRoot; + const bool incomingIsRoot = item.consumerDepIndex == kMainConsumer; + + if (!existingIsRoot && !incomingIsRoot) { + return std::unexpected(std::format( + "dependency '{}{}{}' is requested as both a {} dep " + "(by '{}') and a {} dep (by '{}'). Pick one.\n" + " declare '{}{}{}' in the root to settle it.", + key.ns, key.ns.empty() ? "" : ".", key.shortName, + it->second.source, it->second.requestedBy, + sourceKind, item.requestedBy, + key.ns, key.ns.empty() ? "" : ".", key.shortName)); + } + if (incomingIsRoot && !existingIsRoot) { + // FIFO SEEDING MAKES THIS UNREACHABLE. Every root-declared + // identity is pushed onto `worklist` before this loop + // starts; a transitive dependency's request is pushed + // onto the BACK of the same deque while the loop runs. + // The root's own entry for any identity is therefore + // always dequeued — and resolved — before any + // dependency's request for that identity can arrive. If + // this branch is ever reached, the invariant broke + // upstream (the seed reordered, or a new seed source was + // added after the loop starts): refusing and naming the + // invariant is safer than silently letting whichever side + // arrived first win, which is the accident #630 reports. + return std::unexpected(std::format( + "internal: dependency '{}{}{}': the root's " + "declaration arrived after '{}' had already resolved " + "it. This is unreachable under first-in-first-out " + "worklist seeding; please report this as an mcpp " + "engine defect.", + key.ns, key.ns.empty() ? "" : ".", key.shortName, + it->second.requestedBy)); + } + + // The root already holds this identity (existingIsRoot); the + // incoming, non-root declaration is overridden. When the + // OVERRIDDEN declaration is a version requirement, it is + // still a promise about the graph and is checked against + // what the root's checkout actually is — the same + // Holds/Violated test `addrset::unify` runs for a tool pin + // (address_set.cppm). + if (sourceKind == "version") { + const std::string winnerVersion = it->second.source == "version" + ? it->second.version + : (it->second.depIndex < dep_manifests.size() + ? dep_manifests[it->second.depIndex]->package.version + : std::string{}); + auto req = mcpp::version_req::parse_req(item.originalConstraint); + auto ver = mcpp::version_req::parse_version(winnerVersion); + // An unparseable requirement or checkout version is + // reported as an override below rather than refused: a + // refusal manufactured from ignorance is worse than the + // silent override it would be preventing (the same + // reasoning `addrset::check` states for an unparseable + // spelling). + if (req && ver && !mcpp::version_req::matches(*req, *ver)) { + return std::unexpected(std::format( + "'{}{}{}' is pinned to {} (version {}) by '{}', " + "and '{}' requires {}.\n" + " One checkout of a package is used, so the " + "two cannot both hold.\n" + " fix: relax the requirement, or point the " + "root's pin at a checkout satisfying it.", + key.ns, key.ns.empty() ? "" : ".", key.shortName, + it->second.sourceRef, winnerVersion, + it->second.requestedBy, + item.requestedBy, item.originalConstraint)); + } + } + + mcpp::diag::warning("dependency/source-override", std::format( + "'{}{}{}' is declared as a {} dep (by '{}', {}) and as a " + "{} dep (by '{}', {}); the root's declaration wins.", key.ns, key.ns.empty() ? "" : ".", key.shortName, - it->second.source, it->second.requestedBy, - sourceKind, item.requestedBy)); + it->second.source, it->second.requestedBy, it->second.sourceRef, + sourceKind, item.requestedBy, + sourceKind == "version" ? item.originalConstraint + : sourceRefOf(sourceKind, spec, + item.resolveRoot, + item.originalConstraint)), + std::format("declare '{}{}{}' in the root to choose the other.", + key.ns, key.ns.empty() ? "" : ".", key.shortName)); + + if (it->second.depIndex + 1 < packages.size()) { + recordDependencyEdge(item.consumerDepIndex, + it->second.depIndex + 1, + spec, item.buildOnly); + } + continue; } if (sourceKind == "version" && it->second.version != spec.version) { // SemVer merge attempt: AND-combine the two original @@ -6655,6 +6798,13 @@ prepare_build(bool print_fingerprint, .constraint = item.originalConstraint, .requestedBy = item.requestedBy, .source = "version", + .sourceRef = item.originalConstraint.empty() + ? std::string("*") : item.originalConstraint, + // The mangling fallback refuses a main-package + // participant earlier (see the branch's comment + // above), so this record's requester is always a + // dependency. + .fromRoot = false, .devOnly = item.devOnly, .depIndex = dep_manifests.size() - 1, .linkFlagsAdded = std::move(linkFlagsAdded), @@ -6761,6 +6911,56 @@ prepare_build(bool print_fingerprint, } continue; } + // SAME kind, possibly DIFFERENT reference: two `git` declarations + // of different rev/tag/branch, or two `path` declarations of + // different directories. Row 3 of the decision table (`version` + // vs `version` is handled above and never reaches here). Before + // this comparison existed, the second declaration's reference was + // never even read — the record kept no `path`/`gitRev`, so there + // was nothing to compare, and the winner was whichever request + // happened to be dequeued first (the #630 "accident of queue + // order"). + if (sourceKind != "version") { + const std::string incomingRef = + sourceRefOf(sourceKind, spec, item.resolveRoot, item.originalConstraint); + if (incomingRef != it->second.sourceRef) { + const bool existingIsRoot = it->second.fromRoot; + const bool incomingIsRoot = item.consumerDepIndex == kMainConsumer; + if (incomingIsRoot && !existingIsRoot) { + // See the identical comment in the kind-clash branch + // above: unreachable under FIFO seeding, and refused + // by name rather than silently swapped in. + return std::unexpected(std::format( + "internal: dependency '{}{}{}': the root's " + "declaration arrived after '{}' had already " + "resolved it. This is unreachable under " + "first-in-first-out worklist seeding; please " + "report this as an mcpp engine defect.", + key.ns, key.ns.empty() ? "" : ".", key.shortName, + it->second.requestedBy)); + } + // The already-resolved record wins either way: it is the + // root's (existingIsRoot) or it is simply the first one + // dequeued (neither party is the root). Both are "the + // first requester" in the sense row 3 states — the root + // is dequeued before any transitive request under FIFO + // seeding, so "the root wins" and "the first dequeued + // wins" never disagree about WHICH record already sits in + // `resolved`. + mcpp::diag::warning("dependency/source-override", std::format( + "'{}{}{}' is declared as {} '{}' (by '{}') and as {} " + "'{}' (by '{}'); {} wins.", + key.ns, key.ns.empty() ? "" : ".", key.shortName, + sourceKind, it->second.sourceRef, it->second.requestedBy, + sourceKind, incomingRef, item.requestedBy, + existingIsRoot ? "the root's declaration" + : std::format("'{}', declared first", + it->second.requestedBy)), + std::format("declare '{}{}{}' in the root to choose " + "the other.", + key.ns, key.ns.empty() ? "" : ".", key.shortName)); + } + } // Same key, same version (or compatible path/git) — already // processed; still record the dependency edge before skipping. // Usage propagation is per edge, not per unique package: two @@ -6943,6 +7143,7 @@ prepare_build(bool print_fingerprint, + resolvedGitRev)), }; } + sourceCommit = resolvedGitRev; dep_root = gitRoot; } // (version-source: dep_root + manifest are loaded together via @@ -7080,6 +7281,9 @@ prepare_build(bool print_fingerprint, ? spec.version : dep_manifests.back()->package.version, .sourceKind = sourceKind, + .sourceRef = sourceKind == "git" ? sourceCommit + : sourceKind == "path" ? dep_root.string() + : std::string{}, }); const auto depPackageIndex = packages.size(); packages.push_back(makePackageRoot(dep_root, *dep_manifests.back())); @@ -7093,6 +7297,9 @@ prepare_build(bool print_fingerprint, .constraint = sourceKind == "version" ? item.originalConstraint : "", .requestedBy = item.requestedBy, .source = sourceKind, + .sourceRef = sourceRefOf(sourceKind, spec, item.resolveRoot, + item.originalConstraint), + .fromRoot = item.consumerDepIndex == kMainConsumer, .devOnly = item.devOnly, .depIndex = dep_manifests.size() - 1, .linkFlagsAdded = std::move(linkFlagsAdded), @@ -8723,7 +8930,28 @@ prepare_build(bool print_fingerprint, ? dep_cache_identities[depIdx - 1].indexName : std::string(mcpp::pm::kDefaultNamespace); key.packageName = depName; - key.version = depPkg.manifest.package.version; + // THE VERSION IDENTIFIES THE SOURCES ONLY FOR AN INDEX + // PACKAGE. A `git` package is keyed by its commit and a + // `path` package by a stamp of its tree, because both + // change under an unchanged version and the store then + // serves a binary built from sources that no longer exist + // (#630, item 6; measured 2026-09-08 with examples/12). + // The same rule applies to every upstream below. + auto source_keyed_version = [&](std::size_t pkgIdx) { + const auto& man = packages[pkgIdx].manifest.package; + std::string v = man.version; + if (pkgIdx >= 1 && pkgIdx - 1 < dep_cache_identities.size()) { + const auto& id = dep_cache_identities[pkgIdx - 1]; + if (id.sourceKind == "git" && !id.sourceRef.empty()) + v += "+git." + id.sourceRef; + else if (id.sourceKind == "path") + v += "+path." + mcpp::build::tool_store::tree_stamp( + id.sourceRef.empty() ? packages[pkgIdx].root + : std::filesystem::path(id.sourceRef)); + } + return v; + }; + key.version = source_keyed_version(depIdx); key.targetName = toolName; key.hostTriple = mcpp::toolchain::triple::host_triple().str(); key.compilerIdentity = std::format("{}|{}|{}", @@ -8741,7 +8969,7 @@ prepare_build(bool print_fingerprint, for (auto up : dg::transitive_dependencies(dependencyEdges, depIdx)) key.upstreamKeys.push_back(std::format("{}@{}", packages[up].manifest.package.name, - packages[up].manifest.package.version)); + source_keyed_version(up))); std::ranges::sort(key.upstreamKeys); const auto cacheRoot = mcpp::home::cache_root(); @@ -8757,7 +8985,7 @@ prepare_build(bool print_fingerprint, } mcpp::ui::status("Building", std::format( - "host tool {}:{} from {} v{} (once per package version × " + "host tool {}:{} from {} v{} (once per package source and " "host toolchain)", depName, toolName, depName, depPkg.manifest.package.version)); @@ -9456,6 +9684,37 @@ prepare_build(bool print_fingerprint, // checked against and so the report can show the whole stack. in.compilerFamily = std::string(tc->compiler_family()); in.compilerVersion = tc->version; + // WHETHER THE PAYLOAD HAS A COMPILER RUNTIME FOR AN APPLE CROSS + // TARGET, read from the payload's own resource directory. Clang's + // Darwin driver adds `libclang_rt..a` from there when + // the file exists and continues silently when it does not, and + // the official payload builds only the macOS archive (measured, + // 22.1.8: `lib/clang/22/lib/darwin/` holds `libclang_rt.osx.a` and + // no `ios` or `iossim`). The consequence without this line is + // `__isPlatformVersionAtLeast` undefined at link with nothing + // said earlier (mcpp#630). The engine never looks in Xcode for the + // archive: a compiler runtime the payload lacks is a graph + // package, as it is on the bare rows. + if (!tc->appleSdkRoot.empty()) { + if (auto tt = mcpp::toolchain::triple::parse(tc->targetTriple); + tt && tt->is_ios()) { + const std::string archive = std::format( + "libclang_rt.{}.a", tt->is_ios_simulator() ? "iossim" : "ios"); + const auto payloadRoot = + tc->binaryPath.parent_path().parent_path(); + bool found = false; + std::error_code ec; + for (auto const& ver : std::filesystem::directory_iterator( + payloadRoot / "lib" / "clang", ec)) { + if (std::filesystem::exists( + ver.path() / "lib" / "darwin" / archive, ec)) { + found = true; + break; + } + } + in.payloadCompilerRuntimeAbsent = !found; + } + } } in.compilerRuntime = provider_of(tsd::CapLayer::CompilerRuntime); in.kernelAbi = provider_of(tsd::CapLayer::KernelAbi); @@ -9510,6 +9769,25 @@ prepare_build(bool print_fingerprint, resolvedTargetSide = tsd::resolve(in); targetSideResolved = true; + // REPORTED ONCE, NOT REFUSED. A program that never reaches an + // availability check links and runs without the archive; refusing it + // would trade a diagnosed hazard for a regression. The degradation + // names the platform, the file and the package that supplies it. + if (resolvedTargetSide.compilerRuntime.absent() && tc) { + auto tt = mcpp::toolchain::triple::parse(tc->targetTriple); + mcpp::diag::degraded("target/compiler-runtime", std::format( + "the toolchain payload carries no compiler runtime for {} " + "(no libclang_rt.{}.a under its lib/clang/*/lib/darwin), and no " + "package in the graph provides mcpp:compiler-runtime", + tc->targetTriple, + tt && tt->is_ios_simulator() ? "iossim" : "ios"), + "a program that reaches an availability check " + "(`__builtin_available`, or a system header that uses it) fails " + "at link with `__isPlatformVersionAtLeast` undefined", + "declare `llvm.compiler-rt-builtins` under the target's " + "[target.'cfg(os = \"ios\")'.dependencies]"); + } + // RECORDED ON THE TOOLCHAIN THE MOMENT IT IS KNOWN, because three // producers of a compile line need it and only one of them can see // `resolvedTargetSide`. @@ -10361,10 +10639,17 @@ prepare_build(bool print_fingerprint, // library would be describing something it does not have. for (auto& pkg : packages) { if (pkg.manifest.stdModule.empty()) continue; + // Either spelling of the C++ layer: `hosted-standard-library` is the + // one that predates the layer vocabulary, `mcpp:c++-abi=` the + // current one. A package written against a newer engine may carry + // only the second. const auto& provs = pkg.manifest.provides; - if (std::find(provs.begin(), provs.end(), - std::string{"hosted-standard-library"}) == provs.end()) - continue; + const bool declaresCxxLayer = std::any_of( + provs.begin(), provs.end(), [](const std::string& p) { + return p == "hosted-standard-library" + || p.starts_with("mcpp:c++-abi="); + }); + if (!declaresCxxLayer) continue; auto src = pkg.root / pkg.manifest.stdModule; if (!std::filesystem::exists(src)) { return std::unexpected(std::format( @@ -10429,6 +10714,16 @@ prepare_build(bool print_fingerprint, // DWARF. Same function, not a second copy of the decision. for (auto& f : mcpp::toolchain::graph_runtime_compile_flags(*tc)) flags += " " + f; + // AND THE APPLE CROSS TARGET'S SDK, WHICH THE TOOLCHAIN RESOLUTION + // HAD ALREADY PUT ON THIS CHANNEL AND THIS ASSIGNMENT REPLACES. + // The module's C library is the SDK's on the iOS rows (the + // package supplies the C++ layer alone), and without the sysroot + // the precompile stops on `mbstate_t` inside libc++'s own + // headers. Measured on macos-15 with `llvm.libcxx` over + // `arm64-apple-ios18.0`: twenty "reference to unresolved using + // declaration" errors, every one a C library type. + if (!tc->appleSdkRoot.empty()) + flags += " -isysroot " + mcpp::xlings::shq(tc->appleSdkRoot.string()); } // Everything up to here says which machine the module is for; what // follows says where its headers are. The codegen step needs only the @@ -10490,6 +10785,49 @@ prepare_build(bool print_fingerprint, break; } + // AN APPLE CROSS TARGET WITHOUT A GRAPH C++ RUNTIME LINKS THE SDK'S + // libc++ (the Mach-O cell in distribution.cppm), AND THE HEADERS FOLLOW + // THE RUNTIME. The payload's `std.cppm` and headers describe libc++ 22; + // the SDK's dylib is libc++ 19 (Xcode 16.4, measured), and Apple's SDKs + // ship no module sources of their own (no `usr/share/libc++/v1` on the + // macOS 15.5 and iOS 18.5 SDKs). So: + // + // - a graph that does not import `std` takes the SDK's headers + // (hostflags.cppm, `appleSdkCxxHeaders`): one libc++ on every line, + // and the payload's module, unused, is withdrawn; + // - a graph that imports `std` keeps the payload's module and headers + // over the SDK's dylib. That pairing links until an inline path in + // the newer headers names an export the older dylib lacks + // (`__hash_memory`, `__atomic_notify_all_global_table`, measured), + // and it is what every iOS program built before this release got. + // It is REPORTED ONCE rather than refused: refusing would break a + // program that built yesterday, and the report names the two lines + // that make the hazard disappear. + if (tc && !tc->appleSdkRoot.empty() && targetSideResolved + && !resolvedTargetSide.cxx.fromGraph()) { + if (!needsStdModule) { + tc->appleSdkCxxHeaders = true; + tc->hasImportStd = false; + tc->stdModuleSource.clear(); + tc->stdCompatSource.clear(); + } else { + mcpp::diag::degraded("target/cxx-runtime", std::format( + "{} links the SDK's libc++ under the toolchain payload's " + "libc++ headers and std module, which are a different " + "release of the library", tc->targetTriple), + "the program links while no inline path in the newer headers " + "names an export the SDK's dylib lacks; `std::unordered_map` " + "over `std::string` and `std::atomic::notify_all` are two " + "that do, and they fail at link with `__hash_memory` or " + "`__atomic_notify_all_global_table` undefined", + "declare the C++ standard library as a package, which brings " + "its headers, its module and its objects as one release: " + "[target.'cfg(os = \"ios\")'.dependencies] " + "llvm.libcxx = \"22.1.8.1\" (and " + "llvm.compiler-rt-builtins = \"22.1.8.5\" beside it)"); + } + } + if (needsStdModule && !tc->hasImportStd) { // A freestanding target reaches here for a reason the generic message // gets wrong. Nothing is missing from the toolchain — libc++'s std diff --git a/src/build/prepare_inputs.cppm b/src/build/prepare_inputs.cppm index da61e85d2..76a9977c2 100644 --- a/src/build/prepare_inputs.cppm +++ b/src/build/prepare_inputs.cppm @@ -232,6 +232,15 @@ struct Parser { std::string_view s; std::size_t i = 0; const Ctx& c; std::vector* seenKeys = nullptr; // every `key=` key, in order std::vector* seenWords = nullptr; // every bareword + // Two more optional taps, used only by `os_only_platforms` below and left + // null for `scan_predicate`'s callers so this is additive, not a rewrite. + // `seenKV` carries the VALUE a `seenKeys` entry does not, and `combinator` + // records whether `all`/`any`/`not` fired anywhere in the traversal — an + // OS-only predicate is a single term, and a combinator, even one built + // entirely from OS terms, is not the platform it happens to reduce to on + // this one evaluation. + std::vector>* seenKV = nullptr; + bool* combinator = nullptr; void ws() { while (i < s.size() && std::isspace((unsigned char)s[i])) ++i; } bool eat(char ch) { ws(); if (i < s.size() && s[i] == ch) { ++i; return true; } return false; } std::string ident() { @@ -262,6 +271,7 @@ struct Parser { } bool match_kv(const std::string& k, const std::string& v) { if (seenKeys) seenKeys->push_back(k); + if (seenKV) seenKV->emplace_back(k, v); if (k == "os") return c.os == v; if (k == "arch") return c.arch == v; if (k == "family") return c.family == v; @@ -281,6 +291,7 @@ struct Parser { bool expr() { std::string id = ident(); if (id == "all" || id == "any") { + if (combinator) *combinator = true; eat('('); bool acc = (id == "all"); ws(); @@ -291,7 +302,10 @@ struct Parser { eat(')'); return acc; } - if (id == "not") { eat('('); bool r = expr(); eat(')'); return !r; } + if (id == "not") { + if (combinator) *combinator = true; + eat('('); bool r = expr(); eat(')'); return !r; + } ws(); if (i < s.size() && s[i] == '=') { ++i; return match_kv(id, str()); } return match_alias(id); @@ -360,6 +374,85 @@ inline PredicateScan scan_predicate(const std::string& predicate) { return out; } +// ── #630 item 7: an OS-only selector is a platform ────────────────────────── +// +// `mcpp emit xpkg`'s descriptor has exactly three blocks — `linux`, `macosx`, +// `windows` (`mcpp::pm::emit_xpkg`) — and a `[target.]` predicate is, +// in general, a question about more axes than the descriptor has (arch, env, a +// target-side layer, a feature gate via a combinator). But a predicate that +// asks about NOTHING but the operating system answers a question the +// descriptor already has a block for, so it can be folded into that block +// instead of only producing the `publish/target-axis-tools` advisory. +// +// Built on the SAME `Parser` the evaluator (`matches`) and the diagnostic scan +// (`scan_predicate`) use, not a second reading of the predicate text: it runs +// the one grammar with two more optional taps (`seenKV`, `combinator`) and +// then asks a structural question of the result, rather than pattern-matching +// the source string. A hand-rolled string check here would be a second parser +// of `cfg(...)`, which is the shape this repository has already paid for once +// (`[hooks]` re-parsing mcpp.toml; see the comment on `Parser` above). +// +// Deliberately conservative: a combinator disqualifies the predicate even when +// every operand it combines is itself an OS term. `cfg(any(linux, macos))` is +// true on a broader set of machines than "linux" or "macosx" alone, but the +// descriptor's blocks are per platform, and folding a compound predicate into +// two of them would silently say "installed on this platform" for a predicate +// whose truth also depends on how it combines — `cfg(not(windows))` is the +// case that makes this concrete: it is exactly as OS-only as `cfg(windows)` +// syntactically, and answers a different, unbounded set of platforms (every +// platform this vocabulary does not yet name, not just "macosx and linux"). +// Keeping the warning for every combinator, `not` included, means a predicate +// this function accepts is always a single OS term with no combinator wrapped +// around it — the same seven forms design record 2026-09-13-630 §8.2 lists. +// +// Returns the descriptor block names (`"linux"`, `"macosx"`, `"windows"`) an +// OS-only predicate maps onto, in the same spelling `emit_xpkg` and +// `XlingsConfig::workspaceByPlatform` use; empty for anything else, including +// a predicate this function cannot classify as OS-only at all. +inline std::vector os_only_platforms(const std::string& predicate) { + std::string_view text = predicate; + const bool wrapped = text.starts_with("cfg(") && text.ends_with(")"); + // The bare-alias sugar (`[target.windows]` ≡ `[target.'cfg(windows)']`, + // docs/22) shares the grammar with `cfg(...)` — both are read by the same + // `Parser::expr()` in `matches()` — so both are eligible here. Anything + // else (an exact triple, or the unparsed escape hatch) names neither an + // OS nor a platform on its own. + static constexpr std::string_view kBareAliases[] = { "linux", "macos", "unix", "windows" }; + if (!wrapped && std::ranges::find(kBareAliases, predicate) == std::end(kBareAliases)) + return {}; + std::string_view inner = wrapped ? text.substr(4, text.size() - 5) : text; + + Ctx scratch; + std::vector> kv; + std::vector words; + bool combinator = false; + Parser p{ inner, 0, scratch, nullptr, &words, &kv, &combinator }; + (void)p.expr(); + + // A combinator, or more than one term, disqualifies the predicate — see + // the comment above for why even an all-OS combinator does. + if (combinator || kv.size() + words.size() != 1) return {}; + + if (words.size() == 1) { + // `unix` is the one bareword naming TWO platforms (docs/22: it means + // `c.family == "unix"`, which macOS and Linux both satisfy and Windows + // does not) — matching `matches()`'s own `match_alias`. + if (words[0] == "unix") return { "linux", "macosx" }; + if (words[0] == "linux") return { "linux" }; + if (words[0] == "windows") return { "windows" }; + if (words[0] == "macos") return { "macosx" }; + return {}; // an unrecognised bareword names no platform + } + // kv.size() == 1: only `os = ""` answers a platform; any other key + // (arch, env, a layer, an unrecognised one) is not an OS question. + auto const& [key, value] = kv.front(); + if (key != "os") return {}; + if (value == "linux") return { "linux" }; + if (value == "windows") return { "windows" }; + if (value == "macos") return { "macosx" }; + return {}; // `os = ""` +} + // True when the predicate names a target-side layer and therefore cannot be // answered before dependency resolution. This is the classifier that keeps the // two merge passes disjoint: `append()` is additive, so a section evaluated by diff --git a/src/cli/cmd_publish.cppm b/src/cli/cmd_publish.cppm index 6101a22e6..f5d2a0171 100644 --- a/src/cli/cmd_publish.cppm +++ b/src/cli/cmd_publish.cppm @@ -81,9 +81,12 @@ export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { if (auto v = parsed.value("debug-symbols")) opts.debugSymbols = *v; // `--target` is repeatable: one leg per triple, which is how a library - // package ships for several targets at once. The application path has - // always taken exactly one, and still does — packing one executable for - // several triples would need several executables. + // package ships for several targets at once. The application path takes + // exactly one triple UNLESS the target is a `kind = "app"` whose form is + // a shared object on every requested row (#630 A9, below) — packing an + // executable for several triples would need several executables, and + // that refusal still stands for a `bin` target and for any row whose + // form is not a shared object. std::vector triples; if (auto o = parsed.option("target")) triples = o->get().values; if (!triples.empty()) opts.targetTriple = triples.back(); @@ -122,10 +125,34 @@ export int cmd_pack(const mcpplibs::cmdline::ParsedArgs& parsed) { return mcpp::pack::build_and_pack_library(route->targetName, triples, opts); } if (triples.size() > 1) { - mcpp::ui::error( - "--target may be given once when packing a program: an application " - "bundle wraps one executable, and one executable has one target."); - return 2; + // #630 A9: THE ROUTE IS CHOSEN BY THE ARTIFACT'S FORM, NOT BY THE + // TARGET'S KIND. An `app` whose resolved link form is a shared + // object on EVERY requested row (Android) takes the several-triple + // path a library target already has: the other legs are built first + // (`build_extra_android_legs`, the same leg-loop shape + // `build_and_pack_library` uses) and staged as `lib//` beside + // the primary leg's own `lib//`, into ONE tree, behind ONE + // dispatch. A `bin` target, or any row whose form is an executable, + // keeps today's refusal — packing an executable for several triples + // would need several executables, and there is no "universal + // executable" mechanism the way there is a universal shared object + // (that is `lipo`, a different tool, and out of scope here). + if (!mcpp::pack::accepts_several_targets(*route, triples)) { + mcpp::ui::error( + "--target may be given once when packing a program: an application " + "bundle wraps one executable, and one executable has one target."); + return 2; + } + auto extraLegs = mcpp::pack::build_extra_android_legs( + route->targetName, + std::span(triples).first(triples.size() - 1), + opts.profile); + if (!extraLegs) return 1; // build_extra_android_legs already printed why + // #622 A10: `build_and_pack` now reports the artifact(s) it packed, for + // `mcpp run --format` to take as its operand -- `mcpp pack` itself only + // ever needed the exit code. + return mcpp::pack::build_and_pack(std::move(opts), modeFromUser, + route->targetName, std::move(*extraLegs)).rc; } // #622 A10: `build_and_pack` now reports the artifact(s) it packed, for // `mcpp run --format` to take as its operand -- `mcpp pack` itself only diff --git a/src/pack/binfmt.cppm b/src/pack/binfmt.cppm index 945961f4e..78290bff5 100644 --- a/src/pack/binfmt.cppm +++ b/src/pack/binfmt.cppm @@ -72,6 +72,72 @@ Ident identify(const std::filesystem::path& binary); std::expected, std::string> needed_names(const std::filesystem::path& binary); +// What a Mach-O object's load commands name. +// +// `names`, in load-command order: `LC_LOAD_DYLIB`, `LC_LOAD_WEAK_DYLIB`, +// `LC_REEXPORT_DYLIB` and `LC_LOAD_UPWARD_DYLIB` each contribute one — the +// four load commands that name a dylib this object needs at load time (a +// weak, re-exported or upward one is still a name a bundle has to resolve or +// account for, whatever `dyld` does with it once it is missing). +// `rpaths`, in the same order: `LC_RPATH` search-path entries, which `@rpath` +// in a name (this object's own, or another dylib's already-collected name) is +// resolved against — see `resolve_macho_names`. +// +// A THIN struct rather than growing `needed_names`'s return type: ELF and PE +// resolve entirely through the caller's search directories, so only Mach-O +// has a second list to carry, and `needed_names` keeps returning names alone +// for the two formats that already had callers depending on that shape. +struct MachoNeeded { + std::vector names; + std::vector rpaths; +}; + +// Read `binary`'s Mach-O load commands, or the ones of the SLICE named by +// `arch` when `binary` is a fat (universal) object. +// +// `arch` is mcpp's canonical arch spelling ("x86_64", "aarch64", …) — the +// same spelling `Ident::arch` and the resolved triple already use, so a +// caller passes the triple's arch straight through. A THIN file ignores it: +// there is only one slice to read, and refusing to read it because the +// caller asked for a different architecture would be wrong — the caller +// already knows what it built. An EMPTY `arch` on a fat file reads the FIRST +// slice, which exists so the function is total, not because it is a good +// answer; `needed_names` (below) is the only caller that has no triple to +// pass, and every other caller should supply one. +// +// Handles both endiannesses (the CIGAM magics: a big-endian-authored Mach-O +// is still valid input) and both widths (`mach_header` / `mach_header_64`). +std::expected +macho_needed(const std::filesystem::path& binary, std::string_view arch = {}); + +// One Mach-O dependency NAME, resolved (or not) against the loader's own +// substitution rules — see `resolve_macho_names`. +struct MachoResolved { + std::string name; // as the load command spelled it + std::filesystem::path path; // empty when unresolved + bool unresolved = false; +}; + +// Resolve Mach-O dependency `names` the way `dyld` would, without loading +// anything: `@executable_path` and `@loader_path` are replaced by +// `executableDir` / `loaderDir` wherever either leads a name, and +// `@rpath/` is tried against every entry of `rpaths` IN ORDER — the +// same two substitutions applied to the rpath entry first, then `` +// joined on. The first candidate that exists on disk wins. +// +// A NAME THAT RESOLVES NOWHERE IS REPORTED, NOT DROPPED: the caller decides +// what an unresolved dependency means for the bundle it is building, and +// silently skipping the entry is how one ships without a library it needs. +// +// PURE — every input is a value, the only filesystem access is +// `std::filesystem::exists`, and it is exercised with a name list, an rpath +// list and two directories, never a real Mach-O file. +std::vector +resolve_macho_names(std::span names, + std::span rpaths, + const std::filesystem::path& executableDir, + const std::filesystem::path& loaderDir); + // Is `name` provided by the target OS itself — i.e. must NOT be bundled? // // On ELF this is the manylinux allow-list, which `pack` already had. @@ -160,6 +226,26 @@ std::optional le64(std::string_view b, std::size_t off) { return v; } +// Big-endian counterparts, for the two places this module reads integers +// that are NOT in the reading host's own byte order regardless of platform: +// a FAT Mach-O header (always big-endian on disk, by the format's own +// definition) and a CIGAM (byte-swapped) thin Mach-O's load commands. +std::optional be32(std::string_view b, std::size_t off) { + if (off + 4 > b.size()) return std::nullopt; + std::uint32_t v = 0; + for (int i = 0; i < 4; ++i) + v = (v << 8) | static_cast(b[off + static_cast(i)]); + return v; +} + +std::optional be64(std::string_view b, std::size_t off) { + if (off + 8 > b.size()) return std::nullopt; + std::uint64_t v = 0; + for (int i = 0; i < 8; ++i) + v = (v << 8) | static_cast(b[off + static_cast(i)]); + return v; +} + // Do the bytes at `off` equal `lit`? // // NOT `b.substr(off, n) == lit`, and the difference is a crash. @@ -206,6 +292,158 @@ std::string pe_arch(std::uint16_t machine) { } } +// mach/machine.h `cputype` values for the two slices mcpp's fat Mach-O +// support cares about. Not a general table: a fat binary carrying a slice +// for an architecture mcpp does not build for is a slice this reader never +// has a reason to select. +constexpr std::uint32_t kCpuTypeX86_64 = 0x01000007; +constexpr std::uint32_t kCpuTypeArm64 = 0x0100000c; + +std::string macho_cputype_arch(std::uint32_t cputype) { + switch (cputype) { + case kCpuTypeX86_64: return "x86_64"; + case kCpuTypeArm64: return "aarch64"; + default: return {}; + } +} + +// ─── Mach-O ─────────────────────────────────────────────────────────────── +// +// A thin object's load commands, starting right after the header at `base` +// (0 for a non-fat file, a slice's own offset within `b` for a fat one). +// `bigEndian` says whether THIS SLICE's own integers — cputype (unread +// here), ncmds, and every load command that follows — are stored in the +// opposite byte order from a plain `le32` read; true for the CIGAM magics, +// which `macho_needed` below maps from the four thin magics before calling +// this. +// +// LC_LOAD_DYLIB, LC_LOAD_WEAK_DYLIB, LC_REEXPORT_DYLIB and +// LC_LOAD_UPWARD_DYLIB share one layout after `cmd`/`cmdsize`: an `lc_str` +// at offset 8, a 32-bit OFFSET FROM THE START OF THE LOAD COMMAND (not from +// the file) to a NUL-terminated name. `LC_RPATH` carries the identical +// shape at the identical offset for its path, which is why one read +// (`stringField`) serves every case this function handles. +std::expected +macho_thin_needed(std::string_view b, std::size_t base, bool is64, bool bigEndian) +{ + auto rd32 = [&](std::size_t off) { + return bigEndian ? be32(b, off) : le32(b, off); + }; + + auto ncmds = rd32(base + 16); + auto sizeofcmds = rd32(base + 20); + if (!ncmds || !sizeofcmds) + return std::unexpected("Mach-O header is truncated"); + (void)sizeofcmds; // bounds come from `b`'s own size, not this field. + + const std::size_t cmdsStart = base + (is64 ? 32 : 28); + + constexpr std::uint32_t LC_LOAD_DYLIB = 0x0000000c; + constexpr std::uint32_t LC_LOAD_WEAK_DYLIB = 0x80000018; + constexpr std::uint32_t LC_REEXPORT_DYLIB = 0x8000001f; + constexpr std::uint32_t LC_LOAD_UPWARD_DYLIB= 0x80000023; + constexpr std::uint32_t LC_RPATH = 0x8000001c; + + auto stringField = [&](std::size_t cmdStart) -> std::optional { + auto off = rd32(cmdStart + 8); + if (!off) return std::nullopt; + return cstr(b, cmdStart + *off); + }; + + MachoNeeded out; + std::size_t cursor = cmdsStart; + for (std::uint32_t i = 0; i < *ncmds; ++i) { + auto cmd = rd32(cursor); + auto cmdsize = rd32(cursor + 4); + // A cmdsize too small to hold its own header is malformed input, and + // one of zero would loop forever — both end the walk rather than + // trusting the field. + if (!cmd || !cmdsize || *cmdsize < 8) break; + + if (*cmd == LC_LOAD_DYLIB || *cmd == LC_LOAD_WEAK_DYLIB + || *cmd == LC_REEXPORT_DYLIB || *cmd == LC_LOAD_UPWARD_DYLIB) { + if (auto s = stringField(cursor); s && !s->empty()) + out.names.push_back(std::move(*s)); + } else if (*cmd == LC_RPATH) { + if (auto s = stringField(cursor); s && !s->empty()) + out.rpaths.push_back(std::move(*s)); + } + cursor += *cmdsize; + } + return out; +} + +// Identify the magic at `base` (0, or a fat slice's own offset) and dispatch +// to `macho_thin_needed` with the width and endianness it names. See the +// field comment on `macho_needed` for what the four magics mean. +std::expected +macho_needed_at(std::string_view b, std::size_t base) +{ + if (base + 4 > b.size()) + return std::unexpected("Mach-O slice offset is out of range"); + bool is64, bigEndian; + if (has_at(b, base, std::string_view("\xcf\xfa\xed\xfe", 4))) { is64 = true; bigEndian = false; } + else if (has_at(b, base, std::string_view("\xce\xfa\xed\xfe", 4))) { is64 = false; bigEndian = false; } + else if (has_at(b, base, std::string_view("\xfe\xed\xfa\xcf", 4))) { is64 = true; bigEndian = true; } + else if (has_at(b, base, std::string_view("\xfe\xed\xfa\xce", 4))) { is64 = false; bigEndian = true; } + else return std::unexpected("not a Mach-O object at this offset"); + return macho_thin_needed(b, base, is64, bigEndian); +} + +// Replace a leading `@executable_path` or `@loader_path` with `dir`, +// preserving whatever follows verbatim (typically `/../Frameworks` or +// similar) so `..` is resolved by the filesystem at `exists()`, not by this +// function. `std::nullopt` when `s` does not start with `prefix`. +std::optional +substitute_leading(std::string_view s, std::string_view prefix, + const std::filesystem::path& dir) +{ + if (!s.starts_with(prefix)) return std::nullopt; + return std::filesystem::path(dir.string() + std::string(s.substr(prefix.size()))); +} + +// The byte-level implementation behind the exported `macho_needed`: FAT_MAGIC +// / FAT_MAGIC_64 select a slice by `arch` (or the first slice when `arch` is +// empty — see the exported declaration's comment), everything else is a thin +// object read directly. FAT headers are ALWAYS big-endian on disk regardless +// of a slice's own endianness, which is why `fat_arch`'s fields go through +// `be32`/`be64` unconditionally rather than through `macho_thin_needed`'s +// per-slice `bigEndian` flag. +std::expected +macho_needed(std::string_view b, std::string_view arch) +{ + if (b.size() < 4) return std::unexpected("file is too small to be Mach-O"); + + const bool fat64 = has_at(b, 0, std::string_view("\xca\xfe\xba\xbf", 4)); + if (fat64 || has_at(b, 0, std::string_view("\xca\xfe\xba\xbe", 4))) { + auto nfat = be32(b, 4); + if (!nfat) return std::unexpected("fat Mach-O header is truncated"); + const std::size_t entrySize = fat64 ? 32 : 20; + std::optional chosen, first; + for (std::uint32_t i = 0; i < *nfat; ++i) { + const std::size_t at = 8 + static_cast(i) * entrySize; + auto cputype = be32(b, at); + std::optional offset; + if (fat64) offset = be64(b, at + 8); + else if (auto o = be32(b, at + 8)) offset = *o; + if (!cputype || !offset) + return std::unexpected("fat_arch entry is truncated"); + const auto sliceOffset = static_cast(*offset); + if (!first) first = sliceOffset; + if (!arch.empty() && macho_cputype_arch(*cputype) == arch) { + chosen = sliceOffset; + break; + } + } + auto sliceOffset = chosen ? chosen : first; + if (!sliceOffset) + return std::unexpected("fat Mach-O names no architecture slices"); + return macho_needed_at(b, *sliceOffset); + } + + return macho_needed_at(b, 0); +} + // ─── ELF ──────────────────────────────────────────────────────────────── // // DT_NEEDED lives in the .dynamic section, whose entries are (tag, value) @@ -469,14 +707,15 @@ Ident identify(const std::filesystem::path& binary) { } return id; } - // Mach-O, both endiannesses and the fat wrapper. Recognised but not - // parsed: macOS packaging still asks the loader, and a caller that lands - // here deserves a message naming the format rather than "unknown". + // Mach-O, both endiannesses, and the fat wrapper (32- and 64-bit + // `fat_arch` alike — `needed_names`/`macho_needed` tell those two apart + // by re-reading the same magic; `identify` only has to know it is one). for (auto magic : {std::string_view("\xcf\xfa\xed\xfe", 4), std::string_view("\xce\xfa\xed\xfe", 4), std::string_view("\xfe\xed\xfa\xcf", 4), std::string_view("\xfe\xed\xfa\xce", 4), - std::string_view("\xca\xfe\xba\xbe", 4)}) { + std::string_view("\xca\xfe\xba\xbe", 4), + std::string_view("\xca\xfe\xba\xbf", 4)}) { if (b.starts_with(magic)) { id.format = Format::MachO; return id; } } return id; @@ -491,17 +730,85 @@ needed_names(const std::filesystem::path& binary) { switch (identify(binary).format) { case Format::Elf: return detail::elf_needed(b); case Format::Pe: return detail::pe_needed(b); - case Format::MachO: - return std::unexpected( - "Mach-O dependency reading is not implemented; macOS packaging " - "resolves the closure through the loader instead"); + case Format::MachO: { + // No triple to pass here — see the field comment on the + // exported `macho_needed`'s `arch` parameter. A caller that HAS + // one (`pack::run`'s Mach-O closure step) calls `macho_needed` + // directly instead of through this dispatcher. + auto r = macho_needed(binary, {}); + if (!r) return std::unexpected(r.error()); + return std::move(r->names); + } case Format::Unknown: break; } return std::unexpected(std::format( "'{}' is not an ELF, PE or Mach-O object", binary.string())); } +std::expected +macho_needed(const std::filesystem::path& binary, std::string_view arch) { + auto buf = detail::slurp(binary); + if (!buf) + return std::unexpected(std::format("cannot read '{}'", binary.string())); + return detail::macho_needed(std::string_view{*buf}, arch); +} + +std::vector +resolve_macho_names(std::span names, + std::span rpaths, + const std::filesystem::path& executableDir, + const std::filesystem::path& loaderDir) +{ + auto exists = [](const std::filesystem::path& p) { + std::error_code ec; + return std::filesystem::exists(p, ec) && !ec; + }; + + std::vector out; + for (auto const& name : names) { + MachoResolved r{.name = name}; + std::optional found; + + if (auto p = detail::substitute_leading(name, "@executable_path", executableDir)) { + if (exists(*p)) found = *p; + } else if (auto p = detail::substitute_leading(name, "@loader_path", loaderDir)) { + if (exists(*p)) found = *p; + } else if (name.starts_with("@rpath/")) { + const auto rest = name.substr(std::string_view("@rpath/").size()); + for (auto const& rp : rpaths) { + std::filesystem::path base; + if (auto p = detail::substitute_leading(rp, "@executable_path", executableDir)) + base = *p; + else if (auto p = detail::substitute_leading(rp, "@loader_path", loaderDir)) + base = *p; + else + base = std::filesystem::path(rp); + if (auto candidate = base / rest; exists(candidate)) { + found = candidate; + break; + } + } + } else if (exists(std::filesystem::path(name))) { + // An absolute path, or a bare name found relative to the + // process's own cwd -- tried as-is, same as a name with no + // `@`-prefix at all. + found = std::filesystem::path(name); + } + + if (found) r.path = *found; + else r.unresolved = true; + out.push_back(std::move(r)); + } + return out; +} + bool is_system_lib(Format f, std::string_view name) { + if (f == Format::MachO) { + // Case-sensitive, unlike the PE row below: HFS+/APFS paths are + // (usually) case-sensitive, and `/usr/lib/`/`/System/Library/` are + // the two roots dyld's shared cache and every OS dylib live under. + return name.starts_with("/usr/lib/") || name.starts_with("/System/Library/"); + } std::string lower(name); for (auto& c : lower) c = static_cast(std::tolower(static_cast(c))); diff --git a/src/pack/pack.cppm b/src/pack/pack.cppm index 66724f40e..1ac5d0ad4 100644 --- a/src/pack/pack.cppm +++ b/src/pack/pack.cppm @@ -69,6 +69,26 @@ enum class Mode { None, Static, BundleProject, BundleAll }; // does not control. enum class Format { Tar, Dir, Dispatched }; +// What happens when a FORMAT's dependency closure cannot be resolved on this +// host, as a function of the format alone — never of the reason (a Mach-O +// program whose loader ignores `LD_TRACE_LOADED_OBJECTS`, or a Windows host +// that cannot execute the artifact at all; see the two call sites in `run`). +// +// `Tar` and `Dir` are the archive: the staged tree IS the product, so an +// unavailable closure is the command failing, exactly as before this type +// existed. A dispatched format receives the staged tree regardless, because +// the provider may not need a closure at all — `dist-apk` reads +// `${mcpp.target_file:}` and never looks at the tree — so the outcome +// there is "staged, closure not walked", never a refusal. A pure function of +// one enum value, so it is tested without building a Plan. +enum class ClosureUnavailableOutcome { CommandFails, StageWithoutClosure }; + +ClosureUnavailableOutcome closure_unavailable_outcome(Format format) { + return format == Format::Dispatched + ? ClosureUnavailableOutcome::StageWithoutClosure + : ClosureUnavailableOutcome::CommandFails; +} + struct Options { Mode mode = Mode::BundleProject; Format format = Format::Tar; @@ -190,6 +210,15 @@ struct Plan { // dependency closure attempted -- rather than through the ELF closure // walk below, which asks the file to name its own needs by executing it. bool programIsSharedObject = false; + // #630 A9: the OTHER triples this `app` was packed for, each already + // built by the caller (`pack::pipeline::build_extra_android_legs`) as + // (android abi, its artifact path). Set AFTER `make_plan`, the way + // `strip`/`debugDir` are: what the request came out as once more than + // one `--target` was resolved, which `make_plan` itself has no way to + // know from a single triple. Empty means "an ordinary single-triple + // pack", which is every caller before this item and keeps + // `run_shared_program`'s layout byte-identical for it. + std::vector> extraSharedLegs; // The search set the PE closure resolves names against, after the // contract has had its say (see make_plan). std::vector searchDirs; @@ -205,6 +234,20 @@ struct Plan { struct Error { std::string message; }; +// What step 4 of `run` (resolving the dependency closure) produced. +// +// `walked = false` reaches a caller only for `Format::Dispatched` — +// `closure_unavailable_outcome` turns the same condition into an `Error` for +// `Tar` and `Dir`, so a provider is the only reader that ever sees `false` +// here. `reason` is populated exactly when `!walked`, and is the same text a +// hard refusal used to carry — moved from "before staging" to "step 4's +// outcome", per the design record's decision (§3 of +// 2026-09-13-630-what-a-framework-still-hits-in-the-engine.md). +struct ClosureResult { + bool walked = true; + std::string reason; +}; + // Build a Plan from already-resolved inputs. Caller is expected to have // already run `mcpp build` (or equivalent) and pass the resulting // binary path in. @@ -223,8 +266,11 @@ make_plan(const mcpp::manifest::Manifest& manifest, bool programIsSharedObject = false); // Execute the plan: copies binary + .so + extra files, runs patchelf, -// writes the final tarball or directory. -std::expected +// writes the final tarball or directory. The tree is staged (the program, +// then the declared runtime files) before the dependency closure is +// resolved, so a format whose closure mechanism is unavailable here still +// gets a tree — see `ClosureResult` and `closure_unavailable_outcome`. +std::expected run(const Plan& plan, const mcpp::config::GlobalConfig& cfg); // Helpers used by cli.cppm to render mode names + parse `--mode`. @@ -878,6 +924,78 @@ stage_runtime_files(const Plan& plan, const std::filesystem::path& stagedExeDir) return {}; } +// Steps 1-3 of `run`'s generic (non-PE, non-wasm, non-shared-object) path: +// wipe and recreate the staging root, copy the program into `bin/`, stage the +// files `[runtime] deploy` / `deploy_files` declared (#615), and copy +// README/LICENSE and the host-requirements file when there is one to write. +// +// PORTABLE AND UNCONDITIONAL. Every one of these is a plain filesystem +// operation; none of them executes the artifact and none of them asks what +// the artifact's FORMAT is. That is exactly why it runs before `run` asks +// whether THIS HOST can walk the artifact's dependency closure (step 4, +// below) — the declared files need no loader, only the discovered ones do. +// Moved out of `run` so the Mach-O and Windows-host branches, which used to +// refuse before any of this ran, can stage the same tree a dispatched format +// receives. See §3 of +// .agents/docs/2026-09-13-630-what-a-framework-still-hits-in-the-engine.md. +// +// Returns the path of the staged binary, `/bin/`. +std::expected +stage_declared(const Plan& plan) +{ + std::error_code ec; + std::filesystem::remove_all(plan.stagingRoot, ec); + std::filesystem::create_directories(plan.stagingRoot / "bin", ec); + if (ec) return std::unexpected(Error{std::format( + "cannot create staging '{}': {}", plan.stagingRoot.string(), ec.message())}); + + auto bundledBinary = plan.stagingRoot / "bin" / plan.binaryName; + std::filesystem::copy_file(plan.builtBinary, bundledBinary, + std::filesystem::copy_options::overwrite_existing, ec); + if (ec) return std::unexpected(Error{std::format( + "copy binary failed: {}", ec.message())}); + std::filesystem::permissions(bundledBinary, + std::filesystem::perms::owner_exec + | std::filesystem::perms::group_exec + | std::filesystem::perms::others_exec, + std::filesystem::perm_options::add, ec); + + if (auto r = stage_runtime_files(plan, bundledBinary.parent_path()); !r) + return std::unexpected(r.error()); + + copy_if_exists(plan.projectRoot / "README.md", plan.stagingRoot); + copy_if_exists(plan.projectRoot / "LICENSE", plan.stagingRoot); + + // What the TARGET must provide. Only written when there is something to + // say — an empty file would be read as "nothing is needed", which is a + // claim, and for most programs the absence of the file is the honest + // form of it. When it IS written it is load-bearing: a bundle that omits + // the driver without saying so is a bundle that fails on the user's + // machine with no way to find out why. + if (!plan.hostRequirements.empty()) { + std::ofstream out(plan.stagingRoot / std::filesystem::path(kFileName)); + if (!out) return std::unexpected(Error{std::format( + "cannot write {} into the bundle", kFileName)}); + out << render(plan.hostRequirements); + if (!out) return std::unexpected(Error{std::format( + "failed writing {}", kFileName)}); + } + + return bundledBinary; +} + +// What `run` does with a reason the closure could not be resolved, as a +// function of the plan's requested format — see `closure_unavailable_outcome` +// for the two outcomes and why they differ. +std::expected +finish_without_closure(const Plan& plan, std::string reason) +{ + if (closure_unavailable_outcome(plan.opts.format) + == ClosureUnavailableOutcome::StageWithoutClosure) + return ClosureResult{false, std::move(reason)}; + return std::unexpected(Error{std::move(reason)}); +} + // ─── PE: the closure, read rather than executed ───────────────────────── // // BFS over the import tables, resolving each name against `searchDirs`. A @@ -1155,6 +1273,10 @@ run_wasm(const Plan& plan) // conventionally puts a shared object; a provider that wants the object's // own dependency set bundled (`dist-apk`) reads `${mcpp.target_file:}` // and resolves that itself, out of the engine's closure entirely. +// +// #630 A9: one triple stages flat (`lib/.so`, unchanged); more than +// one triple stages one `lib//.so` per leg into the SAME tree — +// see `Plan::extraSharedLegs`. std::expected run_shared_program(const Plan& plan) { @@ -1164,11 +1286,39 @@ run_shared_program(const Plan& plan) if (ec) return std::unexpected(Error{std::format( "cannot create staging '{}': {}", plan.stagingRoot.string(), ec.message())}); - auto staged = plan.stagingRoot / "lib" / plan.binaryName; - std::filesystem::copy_file(plan.builtBinary, staged, - std::filesystem::copy_options::overwrite_existing, ec); - if (ec) return std::unexpected(Error{std::format( - "copy binary failed: {}", ec.message())}); + // #630 A9: MORE THAN ONE TRIPLE MEANS MORE THAN ONE `lib//`, since a + // flat `lib/.so` cannot hold two architectures' bytes under one + // name. `extraSharedLegs` is non-empty ONLY when the caller is the + // several-`--target` route (`cmd_pack`, via `build_and_pack`'s trailing + // parameter) — every pre-existing single-triple caller leaves it empty + // and keeps the flat layout below byte-identical to before this item. + if (!plan.extraSharedLegs.empty()) { + auto stage_leg = [&](std::string_view abi, const std::filesystem::path& artifact) + -> std::expected + { + auto dir = plan.stagingRoot / "lib" / abi; + std::error_code dec; + std::filesystem::create_directories(dir, dec); + if (dec) return std::unexpected(Error{std::format( + "cannot create staging '{}': {}", dir.string(), dec.message())}); + std::filesystem::copy_file(artifact, dir / plan.binaryName, + std::filesystem::copy_options::overwrite_existing, dec); + if (dec) return std::unexpected(Error{std::format( + "copy binary failed: {}", dec.message())}); + return {}; + }; + auto t = mcpp::toolchain::triple::parse(plan.triple); + auto primaryAbi = t ? mcpp::toolchain::triple::android_abi(*t) : plan.triple; + if (auto r = stage_leg(primaryAbi, plan.builtBinary); !r) return r; + for (auto const& [legAbi, legArtifact] : plan.extraSharedLegs) + if (auto r = stage_leg(legAbi, legArtifact); !r) return r; + } else { + auto staged = plan.stagingRoot / "lib" / plan.binaryName; + std::filesystem::copy_file(plan.builtBinary, staged, + std::filesystem::copy_options::overwrite_existing, ec); + if (ec) return std::unexpected(Error{std::format( + "copy binary failed: {}", ec.message())}); + } // THE RUNTIME FILES TRAVEL AS ON EVERY OTHER ROW. `deploy` placed them // under `bin//` beside the built library; they are staged at the same @@ -1188,7 +1338,7 @@ run_shared_program(const Plan& plan) } // namespace detail -std::expected +std::expected run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) { // A PE package is produced the same way on every host, because nothing in @@ -1200,18 +1350,46 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) // `Elf`, same as any Linux program), so it would otherwise fall into the // closure walk that follows and try to execute an object this host // cannot load at all. See the field comment on `programIsSharedObject`. - if (plan.programIsSharedObject) return detail::run_shared_program(plan); + // + // These three dispatchers read their own closure (PE) or need none at + // all (the Android shared-object row; the wasm32-emscripten launcher, a + // static image with everything embedded) — none of them is the "declare + // before discover" reorder below, so each reports a walked closure on + // success. + if (plan.programIsSharedObject) { + if (auto r = detail::run_shared_program(plan); !r) return std::unexpected(r.error()); + return ClosureResult{}; + } - if (plan.targetIsPe) return detail::run_pe(plan); + if (plan.targetIsPe) { + if (auto r = detail::run_pe(plan); !r) return std::unexpected(r.error()); + return ClosureResult{}; + } // wasm32-emscripten, before the Mach-O refusal and the ELF closure below: // this artifact is neither. `binfmt::identify` reports `Unknown` for the // `.js` launcher (plain text, none of the three magics), which is exactly // the branch the ELF path's own comment warns cannot be assumed away by // exclusion any more. - if (plan.targetIsWasm) return detail::run_wasm(plan); + if (plan.targetIsWasm) { + if (auto r = detail::run_wasm(plan); !r) return std::unexpected(r.error()); + return ClosureResult{}; + } + + using namespace detail; - // A Mach-O artifact is REFUSED, on every host including macOS. + // STEPS 1-3: stage what is DECLARED before asking whether this host can + // WALK what the artifact needs. The program, then the runtime files + // `mcpp::deploy` and `[runtime] deploy` placed beside it, need no + // loader — only the dependency closure below does. See `stage_declared` + // and §3 of + // .agents/docs/2026-09-13-630-what-a-framework-still-hits-in-the-engine.md. + auto staged = stage_declared(plan); + if (!staged) return std::unexpected(staged.error()); + auto bundledBinary = *staged; + + // STEP 4, first mechanism: a Mach-O artifact's closure is never walked, + // on every host including macOS. // // The closure below asks the dynamic linker for the dependency list by // running the artifact with `LD_TRACE_LOADED_OBJECTS=1`. That variable @@ -1237,9 +1415,14 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) // exact defect `route_pack_target` exists to prevent. It is also the only // way an e2e can check that routing on macOS, where no program bundle can // be produced to inspect. + // + // THIS USED TO BE A HARD REFUSAL BEFORE ANY STAGING RAN. It is now step + // 4's outcome: the reason text is unchanged (it is accurate), but for a + // dispatched format the tree staged above is handed to the provider + // regardless — see `finish_without_closure`. if (mcpp::pack::binfmt::identify(plan.builtBinary).format == mcpp::pack::binfmt::Format::MachO) { - return std::unexpected(Error{std::format( + return finish_without_closure(plan, std::format( "cannot package the Mach-O program '{}' yet.\n", plan.binaryName) + " The dependency closure for that format is resolved by running the " "artifact under\n" @@ -1251,17 +1434,21 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) " A `kind = \"lib\"` / `\"shared\"` target packs normally on macOS " "(`mcpp pack `);\n" " for a program, ship the build tree or use a platform bundler until " - "macOS support lands."}); + "macOS support lands."); } #if defined(_WIN32) - // A NON-PE artifact on a Windows host: a cross build to Linux or macOS. - // The closure below asks the dynamic linker by running the binary, which - // this machine cannot do — so say that, rather than reporting a platform - // limitation that no longer exists for the case a Windows user is - // actually likely to hit. + // STEP 4, second mechanism: a NON-PE artifact on a Windows host — a cross + // build to Linux or macOS. The closure would be resolved by running the + // binary under its own dynamic linker, which this machine cannot do — so + // say that, rather than reporting a platform limitation that no longer + // exists for the case a Windows user is actually likely to hit. + // + // Also step 4's outcome now, for the same reason the Mach-O branch above + // is: the tree from `stage_declared` already exists when a dispatched + // format reaches this point. (void)cfg; - return std::unexpected(Error{std::format( + return finish_without_closure(plan, std::format( "cannot package a {} artifact from a Windows host.\n" " The dependency closure for that format is resolved by running " "the artifact under\n" @@ -1271,52 +1458,12 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) "x86_64-pc-windows-msvc`), which\n" " needs no such step.", std::string(mcpp::pack::binfmt::format_name( - mcpp::pack::binfmt::identify(plan.builtBinary).format)))}); + mcpp::pack::binfmt::identify(plan.builtBinary).format)))); #else - using namespace detail; std::error_code ec; - // 1. Wipe + recreate staging dir for a clean snapshot. - std::filesystem::remove_all(plan.stagingRoot, ec); - std::filesystem::create_directories(plan.stagingRoot / "bin", ec); - if (ec) return std::unexpected(Error{std::format( - "cannot create staging '{}': {}", plan.stagingRoot.string(), ec.message())}); - - // 2. Main binary. - auto bundledBinary = plan.stagingRoot / "bin" / plan.binaryName; - std::filesystem::copy_file(plan.builtBinary, bundledBinary, - std::filesystem::copy_options::overwrite_existing, ec); - if (ec) return std::unexpected(Error{std::format( - "copy binary failed: {}", ec.message())}); - std::filesystem::permissions(bundledBinary, - std::filesystem::perms::owner_exec - | std::filesystem::perms::group_exec - | std::filesystem::perms::others_exec, - std::filesystem::perm_options::add, ec); - // 2b. Runtime files beside it, at the paths the build used (#615). - if (auto r = stage_runtime_files(plan, bundledBinary.parent_path()); !r) return r; - - // 3. README / LICENSE if present at project root. - copy_if_exists(plan.projectRoot / "README.md", plan.stagingRoot); - copy_if_exists(plan.projectRoot / "LICENSE", plan.stagingRoot); - - // 3b. What the TARGET must provide. - // - // Only written when there is something to say — an empty file would be - // read as "nothing is needed", which is a claim, and for most programs the - // absence of the file is the honest form of it. When it IS written it is - // load-bearing: a bundle that omits the driver without saying so is a - // bundle that fails on the user's machine with no way to find out why. - if (!plan.hostRequirements.empty()) { - std::ofstream out(plan.stagingRoot / std::filesystem::path(kFileName)); - if (!out) return std::unexpected(Error{std::format( - "cannot write {} into the bundle", kFileName)}); - out << render(plan.hostRequirements); - if (!out) return std::unexpected(Error{std::format( - "failed writing {}", kFileName)}); - } - - // 4. Library bundling for non-static modes. + // STEP 4, third mechanism, and STEP 5 (the format tail): library + // bundling for non-static modes. // // BundleProject (default) — drop all manylinux-allowed system libs // (libc/libstdc++/ld-linux/...) and bundle the rest. The user can @@ -1364,14 +1511,14 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) // that does not fail, it RUNS the module. if (auto fmt = mcpp::pack::binfmt::identify(plan.builtBinary).format; fmt != mcpp::pack::binfmt::Format::Elf) { - return std::unexpected(Error{std::format( + return finish_without_closure(plan, std::format( "cannot package the {} artifact '{}' yet.\n" " Its dependency closure is resolved by running the " "artifact under its own\n" " dynamic linker, and this file is neither ELF, PE nor " "Mach-O -- there is no\n" " such linker to ask.", - mcpp::pack::binfmt::format_name(fmt), plan.binaryName)}); + mcpp::pack::binfmt::format_name(fmt), plan.binaryName)); } auto deps = ldd_parse(plan.builtBinary); if (!deps) return std::unexpected(Error{std::format( @@ -1507,20 +1654,22 @@ run(const Plan& plan, const mcpp::config::GlobalConfig& cfg) return std::unexpected(Error{r.error()}); } - // 4b. Debug information does not travel either. + // The format tail's last two steps, run only because step 4 above + // produced a list (an early return took every path where it did not): + // debug information does not travel either. // // AFTER every byte-changing step above (patchelf's search path, PT_INTERP) // and before the archive: strip must see the final image, and the archive // must see the stripped one. Same ordering rule the library packer states // at its leg loop. - if (auto r = strip_program(plan, bundledBinary); !r) return r; + if (auto r = strip_program(plan, bundledBinary); !r) return std::unexpected(r.error()); - // 5. Output. + // Output. if (plan.opts.format == Format::Tar) { if (auto r = make_tarball(plan.stagingRoot, plan.archivePath); !r) - return r; + return std::unexpected(r.error()); } - return {}; + return ClosureResult{}; #endif // !_WIN32 } diff --git a/src/pack/pipeline.cppm b/src/pack/pipeline.cppm index cf3c87f39..7d10ff49e 100644 --- a/src/pack/pipeline.cppm +++ b/src/pack/pipeline.cppm @@ -41,6 +41,77 @@ export struct PackOutcome { std::vector artifacts; }; +// #630 A9: build every triple but the PRIMARY one for a `kind = "app"` +// target whose form is a shared object on every requested row (Android). +// Each leg is its own `prepare_build` + `ninja` build, the same shape +// `build_and_pack_library`'s leg loop uses and for the same reason: the +// artifact a leg contributes is the one THIS run made, never a glob over +// `target/` that could silently pick up a stale binary from an earlier +// build. The primary triple is not built here — it takes the ordinary +// single-triple `build_and_pack` path, which is what stages the tree, +// deploys the declared files once and runs the one dispatch pass; this +// function only produces the OTHER legs' bytes and where they belong. +// +// Returns `nullopt` on the first leg that fails, having already printed the +// error — the same contract `build_and_pack_library` and `build_and_pack` +// use, so `cmd_pack` only has to check the outcome and return. +export std::optional>> +build_extra_android_legs(const std::string& targetName, + std::span triples, + const std::string& profile) +{ + std::vector> out; + for (auto const& triple : triples) { + mcpp::build::BuildOverrides ov; + ov.target_triple = triple; + // A packaged artifact leaves this machine — same fallback as every + // other pack leg (`build_and_pack_library`, and the primary leg + // below). + ov.profile = profile; + ov.profile_fallback = "release"; + auto ctx = mcpp::build::prepare_build(false, false, {}, ov); + if (!ctx) { mcpp::ui::error(ctx.error()); return std::nullopt; } + + auto be = mcpp::build::make_ninja_backend(); + mcpp::build::BuildOptions bo; + if (auto br = be->build(ctx->plan, bo); !br) { + if (!br.error().diagnosticOutput.empty()) { + std::fputs(br.error().diagnosticOutput.c_str(), stderr); + if (br.error().diagnosticOutput.back() != '\n') std::fputs("\n", stderr); + } + mcpp::ui::error(br.error().message); + return std::nullopt; + } + + // FROM THE PLAN, never a glob — see the function comment. A + // dependency's own `shared` target also contributes a `SharedLibrary` + // link unit to this plan, and `dependencyOwned` is what excludes it + // (the same exclusion `pipeline.cppm`'s `is_program_link_unit` makes + // for the primary leg). + const mcpp::build::LinkUnit* lu = nullptr; + for (auto const& u : ctx->plan.linkUnits) + if (u.targetName == targetName + && u.kind == mcpp::build::LinkUnit::SharedLibrary + && !u.dependencyOwned) { lu = &u; break; } + if (!lu) { + mcpp::ui::error(std::format( + "target '{}' produced no shared-object artifact for {}.\n" + " Every leg of a several-`--target` app pack must resolve to " + "the same shared-object\n" + " form the primary `--target` does; this row did not.", + targetName, triple)); + return std::nullopt; + } + + auto t = mcpp::toolchain::triple::parse(triple); + auto canonical = t ? t->str() : triple; + auto abi = t ? mcpp::toolchain::triple::android_abi(*t) : triple; + mcpp::ui::status("Packed leg", std::format("{} [{}]", canonical, abi)); + out.emplace_back(std::move(abi), ctx->outputDir / lu->output); + } + return out; +} + // Everything after CLI option parsing for `mcpp pack`. // // `wantTarget` is the target NAME the user asked for, empty when they did not. @@ -49,8 +120,21 @@ export struct PackOutcome { // or a project with two `bin` targets would accept `mcpp pack app2` and // silently bundle app1 — the shape where the command succeeds and the answer // is wrong. +// +// `extraLegs` (#630 A9) is every OTHER triple a several-`--target` app-pack +// request named, already built by `build_extra_android_legs` below as +// (android abi, its artifact path). This call still does exactly one build — +// of `opts.targetTriple`, the PRIMARY leg — and stages the primary's own +// artifact and the declared deploy files as it always has; `extraLegs`, when +// non-empty, only changes WHERE the primary's shared object lands +// (`Plan::extraSharedLegs`, read by `run_shared_program`) and adds the other +// legs beside it in the same staged tree, before the one dispatch pass runs. +// Empty for every caller before this item, which is what keeps a +// single-`--target` pack byte-identical. export PackOutcome build_and_pack(Options opts, bool modeFromUser, - const std::string& wantTarget = {}) { + const std::string& wantTarget = {}, + std::vector> + extraLegs = {}) { // `--target *-linux-musl` without an explicit `--mode` implies // `--mode static` — packaging a musl-static ELF as bundle-project // would feed patchelf a static binary and crash. The docs treat @@ -299,6 +383,10 @@ export PackOutcome build_and_pack(Options opts, bool modeFromUser, // which appears in the project's own manifest. ctx->plan.runtimeRequirements, programIsSharedObject); if (!plan) { mcpp::ui::error(plan.error().message); return PackOutcome{1}; } + // #630 A9: see the field comment on `Plan::extraSharedLegs` and the + // parameter comment on `extraLegs` above. A no-op (default-constructed, + // empty) for every caller before this item. + plan->extraSharedLegs = std::move(extraLegs); // The RESOLVED debug-information decision. On the plan, not in Options: // Options is the request, this is what it came out as once the manifest @@ -330,26 +418,31 @@ export PackOutcome build_and_pack(Options opts, bool modeFromUser, // STAGING IS A SERVICE TO THE PROVIDER, NOT A PRECONDITION FOR DISPATCH. // // For `--format tar` and `--format dir` the staged tree IS the product, so - // a staging failure is the command failing. For a DISPATCHED format it is - // an input the provider may or may not want, and treating it as a - // precondition made every dispatched format unreachable on any target - // whose built-in bundling is refused. + // a staging failure -- no tree at all -- is the command failing. For a + // DISPATCHED format a MISSING tree is still failing the same way (a + // genuine I/O error staging steps 1-3), but an UNAVAILABLE CLOSURE is + // not: `pack::run` now stages the program and its declared runtime files + // before it asks whether this host can walk the artifact's dependency + // closure, so a dispatched format receives that tree regardless of + // whether the closure could be resolved. See §3 of + // .agents/docs/2026-09-13-630-what-a-framework-still-hits-in-the-engine.md. // // Measured on macos-15 with mcpp 2026.9.11.1: `mcpp pack --format app` - // never reached the dispatch, because `pack::run` refuses a Mach-O PROGRAM + // never reached the dispatch, because `pack::run` refused a Mach-O PROGRAM // outright -- the built-in closure walk is `LD_TRACE_LOADED_OBJECTS`, which // is glibc's, and dyld ignores it and runs the program instead. That - // refusal is correct about the built-in archive and says nothing about - // whether a `.app` bundler can work, since a bundler that names one + // refusal was correct about the built-in archive and said nothing about + // whether a `.app` bundler could work, since a bundler that names one // program needs no closure walk at all. The engine was answering a // question the provider had not been asked. // - // So the failure is REPORTED AND CARRIED rather than swallowed: the reason - // is printed as a warning, `pack_stage_dir` stays empty, and + // So a missing tree is REPORTED AND CARRIED rather than swallowed: the + // reason is printed as a warning, `pack_stage_dir` stays empty, and // `${mcpp.stage_dir}` then refuses at expansion naming that reason. A // provider that reads the tree gets a precise diagnostic; one that does not // proceeds. Nothing is silently degraded -- what changes is who decides. - std::string stageFailure; + std::string stageFailure; // set only when NO tree exists at all. + mcpp::pack::ClosureStatus closure; if (auto r = mcpp::pack::run(*plan, *cfg); !r) { if (opts.format != mcpp::pack::Format::Dispatched) { mcpp::ui::error(r.error().message); @@ -362,15 +455,27 @@ export PackOutcome build_and_pack(Options opts, bool modeFromUser, "here; one that names a\n" " built file with ${{mcpp.target_file:}} is unaffected.", opts.formatName, stageFailure)); + } else if (!r->walked) { + // The tree exists; only its dependency closure does not. Distinct + // warning text -- "staged" is true here, unlike the branch above. + closure = mcpp::pack::ClosureStatus{false, r->reason}; + mcpp::ui::warning(std::format( + "staged without its dependency closure: {}\n" + " A format that consumes ${{mcpp.stage_dir}} sees the program and its " + "declared\n" + " runtime files but not its discovered dependencies; one that names a " + "built file\n" + " with ${{mcpp.target_file:}} is unaffected.", + closure.reason)); } - // The staged tree is now on disk and final -- past the closure, the - // `$ORIGIN` rewriting, the strip and the debug split. Describe it, so an - // action that consumes it has something whose CONTENT changes when the - // staged set does. Best-effort: see write_stage_manifest. Skipped when - // staging did not happen, so no manifest describes a tree that is not - // there. - if (stageFailure.empty()) mcpp::pack::write_stage_manifest(plan->stagingRoot); + // The staged tree is now on disk and final -- past the closure (walked or + // not), the `$ORIGIN` rewriting, the strip and the debug split. Describe + // it, so an action that consumes it has something whose CONTENT changes + // when the staged set does, and so a provider can read whether the + // closure was walked. Best-effort: see write_stage_manifest. Skipped when + // no tree exists, so no manifest describes a tree that is not there. + if (stageFailure.empty()) mcpp::pack::write_stage_manifest(plan->stagingRoot, closure); auto pathCtx = mcpp::fetcher::make_path_ctx(&*cfg, ctx->projectRoot); diff --git a/src/pack/route.cppm b/src/pack/route.cppm index ab972d8ce..1436f09c0 100644 --- a/src/pack/route.cppm +++ b/src/pack/route.cppm @@ -16,12 +16,20 @@ export module mcpp.pack.route; import std; import mcpp.manifest; import mcpp.project; +import mcpp.toolchain.triple; export namespace mcpp::pack { struct PackRoute { std::string targetName; bool library = false; // kind = lib | shared + // #630 A9: `kind = "app"`, as opposed to `kind = "bin"` — both are + // program routes (`library == false`), but only an `app` can resolve to + // a shared-object FORM on some row (`toolchain::triple:: + // application_form`), which is what lets `mcpp pack` accept more than + // one `--target` for it. `cmd_pack` reads this rather than re-deriving + // it from the manifest a second time. + bool isApplication = false; }; // Resolve `requested` (possibly empty) against the current project. @@ -32,6 +40,20 @@ struct PackRoute { // shape, which is worse than an error. std::expected route_pack_target(std::string_view requested); +// #630 A9: does a several-`--target` pack request suit `route`? The route is +// chosen by the ARTIFACT'S FORM, not by the target's kind: a program route +// serves several triples only when every one of them resolves to a +// shared-object form (Android's `app` rows), which is the library route's +// input already accepts applied to a program. A malformed triple answers +// `false` — the caller reports it as a parse failure, not as a routing +// decision this function made. +// +// A pure function of the route and the requested triples, with no filesystem +// or manifest read of its own, so a routing question this small does not +// need a fixture project to test. +bool accepts_several_targets(const PackRoute& route, + std::span triples); + } // namespace mcpp::pack namespace mcpp::pack { @@ -64,7 +86,8 @@ std::expected route_pack_target(std::string_view request return std::unexpected(std::format( "target '{}' is a test binary; there is nothing to distribute", requested)); - return PackRoute{ t.name, is_library(t) }; + return PackRoute{ t.name, is_library(t), + t.kind == mcpp::manifest::Target::Application }; } std::string list; for (auto const& t : m->targets) { @@ -104,7 +127,9 @@ std::expected route_pack_target(std::string_view request const mcpp::manifest::Target* onlyLib = nullptr; std::size_t libCount = 0; for (auto const& t : m->targets) { - if (t.is_program()) return PackRoute{ t.name, false }; + if (t.is_program()) + return PackRoute{ t.name, false, + t.kind == mcpp::manifest::Target::Application }; if (is_library(t)) { onlyLib = &t; ++libCount; } } if (libCount == 1) return PackRoute{ onlyLib->name, true }; @@ -122,4 +147,17 @@ std::expected route_pack_target(std::string_view request "one for you.\n Name it: {}", list)); } +bool accepts_several_targets(const PackRoute& route, + std::span triples) +{ + if (!route.isApplication) return false; + for (auto const& tr : triples) { + auto t = mcpp::toolchain::triple::parse(tr); + if (!t) return false; + if (mcpp::toolchain::triple::application_form(*t) + != mcpp::toolchain::triple::ApplicationForm::SharedObject) return false; + } + return true; +} + } // namespace mcpp::pack diff --git a/src/pack/stage_tree.cppm b/src/pack/stage_tree.cppm index afc72f461..09089b553 100644 --- a/src/pack/stage_tree.cppm +++ b/src/pack/stage_tree.cppm @@ -30,6 +30,16 @@ // because a staged file whose length is unchanged and whose bytes differ can // only have come from a rebuild, and a rebuild moved the link output that the // dist edge also depends on. +// +// THE HEADER LINE. `mcpp pack` stages the program and its declared runtime +// files before it asks whether this host can walk the artifact's dependency +// closure (§3 of +// .agents/docs/2026-09-13-630-what-a-framework-still-hits-in-the-engine.md), +// so a tree can exist without one -- a Mach-O program today, a non-PE +// artifact on a Windows host. `closure = walked | not-walked` is the one fact +// the manifest states that the sorted file list cannot: a provider that needs +// the closure reads this line and decides for itself rather than discovering +// the gap by what is absent from `lib/`. module; #include @@ -53,6 +63,23 @@ std::filesystem::path stage_manifest_path(const std::filesystem::path& stagingRo return p; } +// Whether `mcpp pack` resolved the staged tree's dependency closure, and why +// not when it did not. +// +// The default (`walked = true`, empty `reason`) is what every archive format +// and every successful ELF/PE pack reports. `walked = false` reaches a +// manifest only for a DISPATCHED format -- `pack::closure_unavailable_outcome` +// turns the same condition into a hard failure for `--format tar` and +// `--format dir`, so those two never write a `not-walked` manifest. +struct ClosureStatus { + bool walked = true; + // Populated only when `!walked`. A SINGLE LINE: the manifest is a plain + // list of one entry per line, and the reason text pack::run produces + // carries its own embedded newlines (it doubles as a CLI diagnostic), so + // `write_stage_manifest` folds them to spaces before writing. + std::string reason; +}; + // Write the manifest for the tree now on disk at `stagingRoot`. // // Best-effort by construction and deliberately so: the manifest is a @@ -61,7 +88,8 @@ std::filesystem::path stage_manifest_path(const std::filesystem::path& stagingRo // manifest makes the dist edge fail with ninja's own "missing and no known rule // to make it", which names the file — a legible failure rather than a silent // staleness. -bool write_stage_manifest(const std::filesystem::path& stagingRoot) { +bool write_stage_manifest(const std::filesystem::path& stagingRoot, + ClosureStatus closure = {}) { std::error_code ec; if (!std::filesystem::is_directory(stagingRoot, ec)) return false; @@ -91,7 +119,27 @@ bool write_stage_manifest(const std::filesystem::path& stagingRoot) { // every run for no reason. std::ranges::sort(lines); - std::string text; + // The header, ahead of the sorted file list and NOT part of it -- it is a + // property of the whole tree, not an entry in it, and mixing the two + // would put "closure = walked" through the same alphabetical sort as a + // path and make its position in the file a function of what got staged. + std::string text = closure.walked ? "closure = walked\n" : "closure = not-walked\n"; + if (!closure.walked) { + // Folded to one line: see the field comment on `ClosureStatus::reason`. + std::string reason = closure.reason; + std::ranges::replace(reason, '\n', ' '); + std::string folded; + bool lastWasSpace = false; + for (char c : reason) { + bool isSpace = (c == ' ' || c == '\t'); + if (isSpace && lastWasSpace) continue; + folded.push_back(isSpace ? ' ' : c); + lastWasSpace = isSpace; + } + while (!folded.empty() && folded.front() == ' ') folded.erase(folded.begin()); + while (!folded.empty() && folded.back() == ' ') folded.pop_back(); + text += std::format("reason = {}\n", folded); + } for (auto const& l : lines) { text += l; text.push_back('\n'); } auto out = stage_manifest_path(stagingRoot); diff --git a/src/pm/publisher.cppm b/src/pm/publisher.cppm index 79cb671d6..206453b40 100644 --- a/src/pm/publisher.cppm +++ b/src/pm/publisher.cppm @@ -12,6 +12,11 @@ import mcpp.manifest; import mcpp.diag; import mcpp.modgraph.graph; import mcpp.platform; +// `cfgpred::os_only_platforms` (#630 item 7) — the standalone half of +// `mcpp.build.prepare`, carrying no dependency on the rest of it (see that +// module's header comment), which is what makes it importable from the +// publish side without pulling in the build plan. +import mcpp.build.prepare_inputs; export namespace mcpp::pm { @@ -198,22 +203,53 @@ std::string emit_xpkg(const mcpp::manifest::Manifest& manifest, out += " type = \"package\",\n\n"; out += " xpm = {\n"; - // A TARGET-AXIS TOOL DECLARATION PRODUCES NO EDGE, AND IS SAID SO HERE. + // A TARGET-AXIS TOOL DECLARATION PRODUCES NO EDGE IN GENERAL, AND IS SAID + // SO HERE -- EXCEPT WHEN THE SELECTOR NAMES ONLY AN OPERATING SYSTEM. // - // `workspaceByPlatform` is filled by the TOP-LEVEL `[xlings.workspace]` - // only, because a descriptor has one block per platform and a platform-keyed + // `workspaceByPlatform` is filled by the TOP-LEVEL `[xlings.workspace]`, + // because a descriptor has one block per platform and a platform-keyed // value is exactly that shape. `[target..xlings.workspace]` is a - // different question -- it conditions on the resolved TARGET, and a - // selector is not a platform: `cfg(target_arch = "aarch64")` names no block - // this file has. + // different question in general -- it conditions on the resolved TARGET, + // and most selectors are not a platform: `cfg(target_arch = "aarch64")` + // names no block this file has, and neither does `cfg(not(windows))` (see + // `os_only_platforms` for why the latter is excluded even though it + // mentions only an OS). // - // Reported rather than dropped, and reported rather than guessed. Mapping a - // selector onto the three blocks would need a representative triple per - // platform, and every entry whose predicate that triple did not satisfy - // would vanish into the same silence this advisory exists to break. The - // same reasoning the superseded `deps` key gets, two comments up. + // But `cfg(linux)`, `cfg(os = "linux")` and their windows/macos/unix + // counterparts DO name a block: they ask the exact question a descriptor + // block answers, on the same axis `[xlings.workspace]` already resolves + // by platform (design record 2026-09-13-630 §8.2). A copy, not the + // manifest's own map, because the merge below is specific to rendering + // this descriptor and must not mutate what the rest of `emit_xpkg` (or a + // second call) reads. + auto byPlatform = manifest.xlings.workspaceByPlatform; + auto merge_into = [&](std::string_view platform, const std::string& address) { + auto& list = byPlatform[std::string(platform)]; + // Do not duplicate an address already present in the block -- an + // `[xlings.workspace]` entry and an OS-only conditional one can name + // the same package without the descriptor listing it twice. + if (std::ranges::find(list, address) == list.end()) + list.push_back(address); + }; for (auto const& cc : manifest.conditionalConfigs) { if (cc.xlings.deps.empty() && cc.xlings.featureDeps.empty()) continue; + auto platforms = mcpp::build::cfgpred::os_only_platforms(cc.predicate); + if (!platforms.empty()) { + // The install-time edge a consumer's `xlings install` needs is + // exactly the same for a package declared here as for one + // declared in the top-level `[xlings.workspace]` restricted to + // this platform -- so it is merged into the same containers, + // under the same key shape (platform → addresses) that table + // fills. Feature-gated tools have no feature axis in this + // descriptor at all yet (the unconditional block above does not + // either), so they fold in unconditionally, same as `deps`. + for (auto const& platform : platforms) { + for (auto const& a : cc.xlings.deps) merge_into(platform, a); + for (auto const& [f, addrs] : cc.xlings.featureDeps) + for (auto const& a : addrs) merge_into(platform, a); + } + continue; + } std::string named; auto add = [&](const std::string& a) { if (!named.empty()) named += ", "; @@ -225,13 +261,14 @@ std::string emit_xpkg(const mcpp::manifest::Manifest& manifest, mcpp::diag::warning("publish/target-axis-tools", std::format( "[target.'{}'] declares tools ({}) and the descriptor carries no " "edge for them: its blocks are per platform, and a selector is not " - "a platform. A consumer of this package installs what the three " - "`xpm..deps` blocks name, which come from the top-level " - "[xlings.workspace]. Declare there anything a CONSUMER must have " - "installed; the target axis stays correct for what this package's " - "own build compiles against.", cc.predicate, named)); + "a platform (an OS-only selector, such as cfg(linux) or " + "cfg(os = \"windows\"), IS emitted as one of the three blocks; " + "this one is not). A consumer of this package installs what the " + "three `xpm..deps` blocks name, which come from the " + "top-level [xlings.workspace]. Declare there anything a CONSUMER " + "must have installed; the target axis stays correct for what " + "this package's own build compiles against.", cc.predicate, named)); } - const auto& byPlatform = manifest.xlings.workspaceByPlatform; out += " linux = {\n" + platform_deps_block(byPlatform, "linux") + platform_block(release.version, release.linux) + " },\n"; out += " macosx = {\n" + platform_deps_block(byPlatform, "macosx") diff --git a/src/toolchain/hostflags.cppm b/src/toolchain/hostflags.cppm index dd9ea0761..af1145311 100644 --- a/src/toolchain/hostflags.cppm +++ b/src/toolchain/hostflags.cppm @@ -136,6 +136,23 @@ struct HostFlagOptions { // every caller that has no graph (the std module build, the build.mcpp // host helper) means. bool cAbiPrebuilt = true; + + // DOES THE C++ RUNTIME COME FROM THE GRAPH? -- `plan.targetSide.cxx.fromGraph()`, + // READ rather than derived from `cAbiPrebuilt`. + // + // The payload's libc++ header set was withheld exactly when the C LIBRARY + // was the graph's. The two questions coincide for openkal (both layers + // from packages) and for a native build (both from the payload), and come + // apart on a hosted target whose C library is a located SDK while a + // package supplies libc++: the iOS rows with `llvm.libcxx` (mcpp#630). + // There the old predicate emitted the payload's `-isystem …/c++/v1` on + // top of the package's headers, two libc++ on one command line. + bool cxxFromGraph = false; + + // THE SDK'S C++ HEADERS INSTEAD OF THE PAYLOAD'S -- `Toolchain::appleSdkCxxHeaders`, + // read rather than derived from `appleSdkRoot`: prepare decides it from + // whether the graph imports `std`, which this function cannot see. + bool appleSdkCxxHeaders = false; }; // Host-compile flags as argv tokens, in the order the string channels have @@ -340,7 +357,17 @@ std::vector host_compile_tokens(const Toolchain& tc, // cannot disagree. const bool graphSuppliesTarget = !opt.cAbiPrebuilt; - if (bypassCfg && !graphSuppliesTarget) { + // THE C++ HEADERS ARE THE C++ LAYER'S QUESTION. `dm.compile_tokens` carries + // libc++'s directories and nothing else, so it is emitted only when the + // payload's libc++ headers are the ones in use: not when a package + // supplies the C++ layer (`cxxFromGraph`), and not when prepare chose + // the SDK's headers for an Apple cross target whose runtime is the SDK's + // libc++ (`appleSdkCxxHeaders`). Measured on Xcode 16.4 with llvm 22.1.8: + // the payload's libc++ 22 headers over the SDK's libc++ 19 dylib fail at + // link on `__hash_memory`, which an inline function in the newer headers + // names and the older dylib does not export (mcpp#630). + const bool cxxFromPayload = !opt.cxxFromGraph && !opt.appleSdkCxxHeaders; + if (bypassCfg && !graphSuppliesTarget && cxxFromPayload) { for (auto& t : dm.compile_tokens(esc, opt.clangStdlibSelect)) out.push_back(t); } else if (bypassCfg) { @@ -367,6 +394,18 @@ std::vector host_compile_tokens(const Toolchain& tc, // Nothing that used to be emitted moves; this path emitted nothing. out.push_back("--no-default-config"); } + if (bypassCfg && !graphSuppliesTarget && !cxxFromPayload) { + // The driver's own C++ search contributes nothing: beside the compiler + // it finds the payload's libc++, and clang's Darwin driver prefers that + // copy to the SDK's whenever it exists. What replaces it is either the + // graph package's directories, which reach every unit through the + // target-side broadcast, or the SDK's `c++/v1`, named here. + out.push_back("-nostdinc++"); + if (opt.clangStdlibSelect) out.push_back("-stdlib=libc++"); + if (!opt.cxxFromGraph) + out.push_back("-isystem" + + esc(opt.appleSdkRoot / "usr" / "include" / "c++" / "v1")); + } // Unconditional on macOS, cfg or no cfg. clang refuses to load a module // built for a different deployment target, and this result feeds every diff --git a/tests/e2e/304_the_target_side_reaches_every_unit.sh b/tests/e2e/304_the_target_side_reaches_every_unit.sh index e62290d6f..7b4d3536d 100755 --- a/tests/e2e/304_the_target_side_reaches_every_unit.sh +++ b/tests/e2e/304_the_target_side_reaches_every_unit.sh @@ -86,11 +86,16 @@ version = "0.1.0" abiprov = { path = "abi" } sibling = { path = "dep" } EOF +# THE C HEADER, NOT THE C++ ONE. `abiprov` declares itself the C++ layer, and +# a package that answers for that layer answers for its headers: the payload's +# libc++ directories are withheld from every unit (#630, item 4). This stub +# ships no standard library, so the probe uses the C library's ``, +# which the payload or the SDK still supplies, rather than ``. cat > src/main.cpp <<'EOF' -#include +#include import abiprov; import sibling; -int main() { std::printf("%d\n", abiprov_v() + sibling_v()); } +int main() { printf("%d\n", abiprov_v() + sibling_v()); } EOF "$MCPP" build >/dev/null 2>&1 || { diff --git a/tests/e2e/330_runner_hosted_targets.sh b/tests/e2e/330_runner_hosted_targets.sh index 91284d48d..f35c0619a 100644 --- a/tests/e2e/330_runner_hosted_targets.sh +++ b/tests/e2e/330_runner_hosted_targets.sh @@ -169,7 +169,7 @@ fi printf '\n[target.%s]\nrunnerX = ["x"]\n' "$HOST" >> mcpp.toml out=$("$MCPP" build 2>&1) grep -q "unsupported key 'runnerX'" <<<"$out" || fail "array typo not reported: $out" -grep -q "Supported keys: cxx_runtime, linkage, runner, sysroot, toolchain" <<<"$out" \ +grep -q "Supported keys: cxx_runtime, linkage, min_api_level, runner, sysroot, toolchain" <<<"$out" \ || fail "runner missing from the supported-keys list: $out" echo "PASS: 330_runner_hosted_targets" diff --git a/tests/e2e/641_the_android_rows_are_wired_and_the_simulator_is_a_row.sh b/tests/e2e/641_the_android_rows_are_wired_and_the_simulator_is_a_row.sh index 2043fa685..db7ab12d1 100755 --- a/tests/e2e/641_the_android_rows_are_wired_and_the_simulator_is_a_row.sh +++ b/tests/e2e/641_the_android_rows_are_wired_and_the_simulator_is_a_row.sh @@ -285,5 +285,30 @@ done # tests/unit/test_toolchain_triple.cpp, where `llvm_triple` is asked directly # and every host can ask it. +# 9. `min_api_level` DECLARED ON ONE ROW MUST NOT TRIP `--strict` WHEN +# BUILDING A DIFFERENT ROW. The unknown-key sweep runs over every +# `[target.]` section in the manifest regardless of which target +# is resolved, so a manifest that declares `min_api_level` only under +# `[target.aarch64-linux-android]` and is then built for this (non-Android) +# host must produce neither a schema warning nor, under `--strict`, a +# failure. Before #610's drift was closed, this exact shape failed: +# `min_api_level` was parsed and honoured for the android row yet reported +# as "has unsupported key 'min_api_level'" for it, and `--strict` turned +# that report into a build failure that had nothing to do with the target +# actually being built. +d="$t/apilevel-strict-other-row"; pkg "$d" "" "[target.aarch64-linux-android]" "min_api_level = 24" +out=$( cd "$d" && "$MCPP" build --strict 2>&1 ); rc=$? +if [ "$rc" -ne 0 ]; then + echo "FAIL: --strict build for this host failed with min_api_level declared on another row" + grep -m5 -E "^error|unsupported key" <<<"$out" | sed 's/^/ /' + fail=1 +elif grep -qi "unsupported key" <<<"$out"; then + echo "FAIL: --strict build reported min_api_level as an unsupported key" + grep -i "unsupported key" <<<"$out" | sed 's/^/ /' + fail=1 +else + echo " ok: min_api_level on [target.aarch64-linux-android] does not trip --strict for this row" +fi + if [ "$fail" -ne 0 ]; then echo "FAIL: 641"; exit 1; fi echo "PASS: 641" diff --git a/tests/e2e/661_one_identity_two_declarations.sh b/tests/e2e/661_one_identity_two_declarations.sh new file mode 100755 index 000000000..5766086c2 --- /dev/null +++ b/tests/e2e/661_one_identity_two_declarations.sh @@ -0,0 +1,448 @@ +#!/usr/bin/env bash +# requires: gcc +# 661_one_identity_two_declarations.sh -- two declarations of the SAME +# dependency identity (T3, 2026-09-13-630 record §2.2). Before this, a +# second `git`/`path` declaration of an already-resolved identity carried +# no comparable reference at all -- `ResolvedRecord` held no path and no +# `gitRev` -- so the winner was silently whichever request the FIFO +# worklist happened to dequeue first, and a root/dependency KIND clash +# (`path` vs `git`) was an unconditional refusal even when the root itself +# was a party. Six cases: +# +# 1. root `git rev=A`, a library `git rev=B` -> root wins, warned +# 2. root `path` (dirty working tree), library `git` -> root wins, warned +# 3. no root declaration, two libraries at B and C -> first wins, warned +# 4. two libraries, one `path` one `git` -> refused, "Pick one" +# 5. root pins a checkout below a library's SemVer floor -> refused +# 6. the SAME reference declared twice -> silent, --strict OK +set -e +source "$(dirname "$0")/_host_path.sh" + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT + +fail() { + local msg="$1"; shift + echo "FAIL: $msg" + for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done + exit 1 +} + +# ── one shared git origin for "framework": commits A, B, C, then a dirty +# working tree. A `git clone` of a commit never sees the dirty edit made +# afterwards, so cases 1/3/4/6 (which clone a fixed rev) and case 2 (which +# reads the live directory as a `path` dep) can share this one repository. ── +FW_GIT="$TMP/framework-git" +FW_GIT_HOST="$(host_path "$FW_GIT")" +mkdir -p "$FW_GIT/src" +git init --quiet "$FW_GIT" +git -C "$FW_GIT" config user.email "test@local" +git -C "$FW_GIT" config user.name "test" +cat > "$FW_GIT/mcpp.toml" <<'EOF' +[package] +name = "framework" +version = "0.1.0" + +[build] +sources = ["src/*.c"] + +[targets.framework] +kind = "lib" +EOF + +write_marker() { + printf 'int framework_marker(void) { return %s; }\n' "$1" > "$FW_GIT/src/framework.c" +} + +write_marker 101 +git -C "$FW_GIT" add -A >/dev/null +git -C "$FW_GIT" commit --quiet -m "A" +REV_A=$(git -C "$FW_GIT" rev-parse HEAD) + +write_marker 102 +git -C "$FW_GIT" add -A >/dev/null +git -C "$FW_GIT" commit --quiet -m "B" +REV_B=$(git -C "$FW_GIT" rev-parse HEAD) + +write_marker 103 +git -C "$FW_GIT" add -A >/dev/null +git -C "$FW_GIT" commit --quiet -m "C" +REV_C=$(git -C "$FW_GIT" rev-parse HEAD) + +# The dirty edit for case 2 -- never committed, so REV_A/B/C above are +# unaffected by it. +write_marker 199 + +# A second "framework", pinned by a `path` dep and carrying its own +# `[package] version` -- used by case 5's SemVer-against-checkout check. +FW_V020="$TMP/framework-v020" +mkdir -p "$FW_V020/src" +cat > "$FW_V020/mcpp.toml" <<'EOF' +[package] +name = "framework" +version = "0.2.0" + +[build] +sources = ["src/*.c"] + +[targets.framework] +kind = "lib" +EOF +printf 'int framework_marker(void) { return 20; }\n' > "$FW_V020/src/framework.c" + +# make_lib DIR NAME EXTRA_MANIFEST_LINES -- a tiny library whose one +# function calls framework_marker() and returns it, so a consumer can +# observe WHICH framework instance the library actually built against. +make_lib() { + local dir="$1" name="$2" extra="$3" + mkdir -p "$dir/src" + { + printf '[package]\nname = "%s"\nversion = "0.1.0"\n\n' "$name" + printf '[build]\nsources = ["src/*.c"]\n\n' + printf '[targets.%s]\nkind = "lib"\n\n' "$name" + printf '%s\n' "$extra" + } > "$dir/mcpp.toml" + printf 'extern int framework_marker(void);\nint %s_marker(void) { return framework_marker(); }\n' \ + "$name" > "$dir/src/$name.c" +} + +# ═══════════════════════════════════════════════════════════════════════ +# Case 1: root `git rev=A`, a library `git rev=B` -- the root wins. +# ═══════════════════════════════════════════════════════════════════════ +C1="$TMP/case1" +make_lib "$C1/libb" "libb" "$(cat < "$C1/app/mcpp.toml" < "$C1/app/src/main.cpp" <<'EOF' +#include +extern "C" int libb_marker(void); +int main() { + std::printf("libb=%d\n", libb_marker()); + return 0; +} +EOF +( + cd "$C1/app" + ec=0 + "$MCPP" run > run.log 2>&1 || ec=$? + [[ $ec -eq 0 ]] || fail "case 1: exit $ec" run.log + grep -q "app" run.log || fail "case 1: warning does not name the root" run.log + grep -q "libb" run.log || fail "case 1: warning does not name the library" run.log + grep -q "$REV_A" run.log || fail "case 1: warning does not name rev A" run.log + grep -q "$REV_B" run.log || fail "case 1: warning does not name rev B" run.log + grep -q "libb=101" run.log || fail "case 1: program did not print A's marker" run.log + grep -q "$REV_A" mcpp.lock || fail "case 1: mcpp.lock does not record A" mcpp.lock +) +echo "ok: case 1 -- root git rev wins over a library's, warned, locked, built against A" + +# ═══════════════════════════════════════════════════════════════════════ +# Case 2: root `path` (dirty working tree), a library `git` -- root wins, +# and what the library actually sees is the WORKING TREE, not a commit. +# ═══════════════════════════════════════════════════════════════════════ +C2="$TMP/case2" +make_lib "$C2/libg" "libg" "$(cat < "$C2/app/mcpp.toml" < "$C2/app/src/main.cpp" <<'EOF' +#include +extern "C" int libg_marker(void); +int main() { + std::printf("libg=%d\n", libg_marker()); + return 0; +} +EOF +( + cd "$C2/app" + ec=0 + "$MCPP" run > run.log 2>&1 || ec=$? + [[ $ec -eq 0 ]] || fail "case 2: exit $ec" run.log + grep -q "libg" run.log || fail "case 2: warning does not name the library" run.log + grep -q "libg=199" run.log \ + || fail "case 2: program did not print the working tree's marker" run.log +) +echo "ok: case 2 -- root path wins over a library's git rev, program sees the working tree" + +# ═══════════════════════════════════════════════════════════════════════ +# Case 3: no root declaration, two libraries at B and C -- the first one +# dequeued (libb, alphabetically before libc) wins, and BOTH edges are +# redirected to it -- libc_marker() must also read B, not C. +# ═══════════════════════════════════════════════════════════════════════ +C3="$TMP/case3" +make_lib "$C3/libb" "libb" "$(cat < "$C3/app/mcpp.toml" <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[build] +sources = ["src/*.cpp"] + +[targets.app] +kind = "bin" +main = "src/main.cpp" + +[dependencies.libb] +path = "../libb" + +[dependencies.libc] +path = "../libc" +EOF +cat > "$C3/app/src/main.cpp" <<'EOF' +#include +extern "C" int libb_marker(void); +extern "C" int libc_marker(void); +int main() { + std::printf("libb=%d libc=%d\n", libb_marker(), libc_marker()); + return 0; +} +EOF +( + cd "$C3/app" + ec=0 + "$MCPP" run > run.log 2>&1 || ec=$? + [[ $ec -eq 0 ]] || fail "case 3: exit $ec" run.log + grep -q "libb" run.log || fail "case 3: warning does not name libb" run.log + grep -q "libc" run.log || fail "case 3: warning does not name libc" run.log + grep -q "libb=102 libc=102" run.log \ + || fail "case 3: both libraries did not converge on the first-resolved framework (B)" run.log +) +echo "ok: case 3 -- with no root opinion, the first-dequeued declaration wins for both edges" + +# ═══════════════════════════════════════════════════════════════════════ +# Case 4: two libraries, one `path` one `git` -- neither is the root, so +# this is refused exactly as a KIND clash always was, plus the new hint. +# ═══════════════════════════════════════════════════════════════════════ +C4="$TMP/case4" +FW_V020_HOST="$(host_path "$FW_V020")" +make_lib "$C4/libd" "libd" "$(cat < "$C4/app/mcpp.toml" <<'EOF' +[package] +name = "app" +version = "0.1.0" + +[build] +sources = ["src/*.cpp"] + +[targets.app] +kind = "bin" +main = "src/main.cpp" + +[dependencies.libd] +path = "../libd" + +[dependencies.libe] +path = "../libe" +EOF +cat > "$C4/app/src/main.cpp" <<'EOF' +int main() { return 0; } +EOF +( + cd "$C4/app" + "$MCPP" build > build.log 2>&1 && ec=0 || ec=$? + [[ $ec -ne 0 ]] || fail "case 4: expected a refusal but the build succeeded" build.log + grep -q "Pick one" build.log || fail "case 4: refusal message missing 'Pick one'" build.log + grep -q "in the root to settle it" build.log \ + || fail "case 4: refusal is missing the new root hint" build.log + grep -q "libd" build.log || fail "case 4: refusal does not name libd" build.log + grep -q "libe" build.log || fail "case 4: refusal does not name libe" build.log +) +echo "ok: case 4 -- a kind clash between two non-root requesters is still refused, with the hint" + +# ═══════════════════════════════════════════════════════════════════════ +# Case 5: the root pins a checkout below a library's SemVer floor -- the +# root still wins the reference, but the floor is checked against the +# checkout's OWN `[package] version` and a violation is refused. +# ═══════════════════════════════════════════════════════════════════════ +C5="$TMP/case5" +INDEX_DIR="$C5/local-index" +INDEX_DIR_HOST="$(host_path "$INDEX_DIR")" +mkdir -p "$INDEX_DIR/pkgs/f" +cat > "$INDEX_DIR/pkgs/f/framework.lua" <<'EOF' +package = { + spec = "1", + name = "framework", + description = "SemVer side of the 661 fixture -- never actually fetched", + licenses = {"MIT"}, + type = "package", + xpm = { + linux = { + ["0.5.0"] = { + url = "https://example.invalid/framework-0.5.0.tar.gz", + sha256 = "0000000000000000000000000000000000000000000000000000000000000000", + }, + }, + }, + mcpp = { + language = "c++23", + import_std = false, + sources = { "src/*.c" }, + targets = { ["framework"] = { kind = "lib" } }, + deps = {}, + }, +} +EOF +make_lib "$C5/libv" "libv" "$(cat <<'EOF' +[dependencies] +framework = ">=0.3" +EOF +)" +mkdir -p "$C5/app/src" +cat > "$C5/app/mcpp.toml" < "$C5/app/src/main.cpp" <<'EOF' +int main() { return 0; } +EOF +( + cd "$C5/app" + "$MCPP" build > build.log 2>&1 && ec=0 || ec=$? + [[ $ec -ne 0 ]] || fail "case 5: expected a refusal but the build succeeded" build.log + grep -q "0.2.0" build.log || fail "case 5: refusal does not name the checkout's version" build.log + grep -q ">=0.3" build.log || fail "case 5: refusal does not name the requirement" build.log + grep -q "app" build.log || fail "case 5: refusal does not name the root" build.log + grep -q "libv" build.log || fail "case 5: refusal does not name the library" build.log +) +echo "ok: case 5 -- a root pin below a library's SemVer floor is refused, naming both" + +# ═══════════════════════════════════════════════════════════════════════ +# Case 6 (negative): the SAME reference declared twice must warn about +# NOTHING, and must not trip `--strict` -- without this row, a fixture +# that warns unconditionally would still pass every row above. +# ═══════════════════════════════════════════════════════════════════════ +C6="$TMP/case6" +make_lib "$C6/libf" "libf" "$(cat < "$C6/app/mcpp.toml" < "$C6/app/src/main.cpp" <<'EOF' +#include +extern "C" int libf_marker(void); +int main() { + std::printf("libf=%d\n", libf_marker()); + return 0; +} +EOF +( + cd "$C6/app" + ec=0 + "$MCPP" build --strict > build.log 2>&1 || ec=$? + [[ $ec -eq 0 ]] || fail "case 6: --strict refused an identical, repeated declaration" build.log + # `grep -q ... && fail ...` would make a NON-match (the success path here) + # the last command's exit status, which -- under `set -e`, outside any + # conditional -- would abort the whole script silently. `if` is what + # keeps a passing check from reading as a script failure. + if grep -q "wins" build.log; then + fail "case 6: an identical reference declared twice still produced an override warning" build.log + fi +) +echo "ok: case 6 -- the same reference declared twice is silent, and --strict stays green" + +echo "OK" diff --git a/tests/e2e/662_pack_stages_declared_files_before_the_closure.sh b/tests/e2e/662_pack_stages_declared_files_before_the_closure.sh new file mode 100755 index 000000000..2ad8c7de1 --- /dev/null +++ b/tests/e2e/662_pack_stages_declared_files_before_the_closure.sh @@ -0,0 +1,160 @@ +#!/usr/bin/env bash +# requires: gcc +# 662_pack_stages_declared_files_before_the_closure.sh -- `mcpp pack` stages +# the program and its declared runtime files BEFORE it asks whether this +# host can walk the artifact's dependency closure, and records the outcome +# of that closure walk on the stage manifest. +# +# See .agents/docs/2026-09-13-630-what-a-framework-still-hits-in-the-engine.md +# §3. Before this, a program whose closure could not be walked (a Mach-O on +# any host, a non-PE artifact on a Windows host) was refused BEFORE anything +# was staged, so a dispatched format that names one program and never reads +# the closure (an `.app`/`.msi` bundler) had no tree to work from either. +# Reordering the steps gives every dispatched format a tree regardless, and +# leaves the two ELF-native formats (`tar`, `dir`) exactly as they were: the +# staged/archived tree IS the product for those two, so an unavailable +# closure remains the command failing. +# +# TWO LEGS ON THIS (LINUX) HOST, EACH WITH ITS OWN CLAIM: +# +# A. `--format dir` of an ordinary ELF program: the reorder must not change +# the ELF product. Asserted as an exact structure -- `bin/`, +# the declared runtime file, an empty `lib/` (this fixture bundles +# nothing), and a stage manifest that says `closure = walked` -- rather +# than "the command exited 0", which a wrong reorder also satisfies. +# B. A DISPATCHED format (`--format zap`, in the shape 638 uses) whose +# action reads `${mcpp.stage_dir}`: the tree it sees carries the +# declared runtime file, and the manifest beside it says +# `closure = walked` too -- on Linux the ELF closure is always walked, +# so this leg's job is to prove the field REACHES a provider, not that +# it ever reads "not-walked" here. +# +# THE "NOT-WALKED" CASE CANNOT BE PRODUCED ON LINUX. It needs a Mach-O +# program (refused on every host, by format) or a non-PE artifact packed +# from a Windows host -- neither is reachable from a `gcc`-only Linux runner. +# It is covered instead by: +# - a unit test of the pure decision function +# (`PackClosureUnavailableOutcome` in test_pack_modes.cpp) and of the +# manifest writer/reader round-trip for both values +# (`PackStageTree.*Closure*` in test_pack_stage_tree.cpp); +# - 266_pack_refuses_a_macho_program.sh (`# requires: macos`), which holds +# the negative direction end to end: `mcpp pack` (bare, i.e. `--format +# tar`) of a Mach-O program still exits non-zero. +# The macOS half of THIS fixture -- a dispatched format on a Mach-O program +# staging without its closure, with `closure = not-walked` on the manifest +# and the reason readable by the provider -- is a follow-up for whoever runs +# this suite's iOS/macOS CI job; it is not asserted by any step here. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +export MCPP_HOME=$HOME/.mcpp + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +# ── Leg A: `--format dir`, byte-for-byte the same ELF product ────────────── +cd "$TMP" +"$MCPP" new app > /dev/null +cd app +mkdir -p share +printf 'notes\n' > share/notes.txt +cat >> mcpp.toml <<'EOF' + +[toolchain] +linux = "gcc@16.1.0" + +[runtime] +deploy_files = ["share/notes.txt"] +EOF + +"$MCPP" pack --format dir > dir.log 2>&1 || fail "mcpp pack --format dir failed" dir.log + +tree=$(find target/dist -maxdepth 1 -type d -name 'app-0.1.0-*' | head -1) +[ -n "$tree" ] || fail "no staged directory under target/dist" dir.log +[ -x "$tree/bin/app" ] || fail "the staged tree has no executable bin/app" dir.log +[ -f "$tree/bin/notes.txt" ] || fail "the declared runtime file did not reach bin/" dir.log +grep -q 'notes' "$tree/bin/notes.txt" || fail "the deployed file's content is wrong" dir.log +# No third-party dependency was declared, so nothing should be bundled -- +# the reorder must not manufacture a lib/ entry that was not there before it. +if [ -d "$tree/lib" ]; then + n=$(find "$tree/lib" -mindepth 1 | wc -l) + [ "$n" -eq 0 ] || { find "$tree/lib"; fail "lib/ has entries but nothing was declared" dir.log; } +fi +"$tree/bin/app" > run.log 2>&1 || fail "the staged program does not run" run.log dir.log + +manifest="$(dirname "$tree")/$(basename "$tree").stage-manifest" +[ -f "$manifest" ] || fail "no stage manifest beside the staged tree" dir.log +[ "$(sed -n '1p' "$manifest")" = "closure = walked" ] \ + || fail "the manifest's first line is not 'closure = walked'" "$manifest" dir.log +grep -q '^reason = ' "$manifest" \ + && fail "a walked closure must carry no reason line" "$manifest" +grep -q ' bin/notes.txt$' "$manifest" || fail "the deploy file is not in the manifest" "$manifest" +echo " leg A: --format dir stages bin/app + the deploy file, closure = walked" + +# ── Leg B: a dispatched format reads the same tree + the same field ──────── +cd "$TMP" +"$MCPP" new zapapp > /dev/null +cd zapapp +mkdir -p share +printf 'notes\n' > share/notes.txt +cat >> mcpp.toml <<'EOF' + +[toolchain] +linux = "gcc@16.1.0" + +[runtime] +deploy_files = ["share/notes.txt"] +EOF + +cat > dist.sh <<'EOF' +#!/usr/bin/env bash +set -e +stage="$1"; manifest="$2"; out="$3" +{ + echo "staged:" + ls -1 "$stage/bin" | sort + echo "manifest-first-line:" + sed -n '1p' "$manifest" +} > "$out" +EOF +chmod +x dist.sh + +cat > build.mcpp <<'EOF' +import mcpp; +#include +#include +int main() { + // Unconditional half: this build knows a "zap" format exists whether or + // not one was requested (638 states why). + mcpp::provides_pack_format("zap"); + if (std::string_view(mcpp::pack_format()) != "zap") return 0; + + const std::string root = mcpp::manifest_dir(); + const std::string stage = std::string("${mcpp.stage_dir}"); + const std::string out = std::string(mcpp::out_dir()) + "/app.zap"; + mcpp::action a; + a.id = "zap"; + a.role = "artifact"; + a.description = "zap"; + a.arg((root + "/dist.sh").c_str()) + .arg(stage.c_str()) + .arg((stage + ".stage-manifest").c_str()) + .arg(out.c_str()) + .input("${mcpp.target_file:zapapp}") + .output(out.c_str()) + .submit(); + return 0; +} +EOF + +"$MCPP" pack --format zap > zap.log 2>&1 || fail "mcpp pack --format zap failed" zap.log +Z=$(find target -name 'app.zap' | head -1) +[ -n "$Z" ] || fail "--format zap produced nothing" zap.log + +grep -qx "notes.txt" "$Z" \ + || fail "the action's own view of the staged tree lacks the deploy file" "$Z" zap.log +grep -qx "closure = walked" "$Z" \ + || fail "the manifest the action read does not say closure = walked" "$Z" zap.log +echo " leg B: --format zap's action sees the deploy file and closure = walked" + +echo "PASS: 662_pack_stages_declared_files_before_the_closure" diff --git a/tests/e2e/663_a_graph_libcxx_over_the_payloads_c_library.sh b/tests/e2e/663_a_graph_libcxx_over_the_payloads_c_library.sh new file mode 100755 index 000000000..a34424be0 --- /dev/null +++ b/tests/e2e/663_a_graph_libcxx_over_the_payloads_c_library.sh @@ -0,0 +1,75 @@ +#!/usr/bin/env bash +# requires: llvm +# 663 -- a package supplies the C++ standard library while the C library stays +# the payload's (#630, item 4). `llvm.libcxx` carries libc++ and libc++abi as +# source with a std module; the engine reports the C++ layer as the graph's, +# withholds the payload's libc++ headers, links with -nostdlib++, and the +# program runs with no libc++ shared object in its closure. This is the +# combination the iOS rows need, measured here on Linux over glibc, which is +# the one host the shards have. The negative direction: the same program +# without the declaration keeps the payload's headers and its command lines. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +export MCPP_HOME=$HOME/.mcpp + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +cd "$TMP" +"$MCPP" new app > /dev/null +cd app +cat > src/main.cpp <<'CPP' +import std; +int main() { + std::unordered_map m; + m["one"] = 1; + std::atomic a{2}; + a.notify_all(); + std::print("{}-{}-3\n", m["one"], a.load()); +} +CPP +cat >> mcpp.toml <<'TOML' + +[toolchain] +default = "llvm@22.1.8" +TOML + +# The negative direction first, so that the baseline is read before the +# package changes anything: the payload's headers and its own runtime. +"$MCPP" build > base.log 2>&1 || fail "the baseline build failed" base.log +# The report prints a layer only when a package answers for it, so the +# baseline is read from the command lines rather than from a line that is +# absent by design. +grep -q 'c++-abi.*graph)' base.log && fail "the baseline report names a graph C++ layer" base.log +base_ninja=$(ls target/*/*/build.ninja | head -1) +grep -q -- '-isystem.*include/c++/v1' "$base_ninja" \ + || fail "the baseline compile line carries no payload libc++ -isystem" "$base_ninja" +# On this host the payload's contract is self-contained, so the link line +# names the payload's own archives; that, not `-nostdlib++`, is the mark of +# the payload's runtime. +grep -q -- 'xim-x-llvm[^ ]*libc++\.a' "$base_ninja" \ + || fail "the baseline link line does not name the payload's libc++.a" "$base_ninja" +echo "ok: without the package the payload's libc++ is used" + +cat >> mcpp.toml <<'TOML' + +[dependencies] +llvm.libcxx = { git = "https://github.com/mcpplibs/libcxx.git", tag = "22.1.8.1" } +TOML +rm -rf target +"$MCPP" build > build.log 2>&1 || fail "the build over llvm.libcxx failed" build.log +grep -E 'c\+\+-abi +libc\+\+ +\(libcxx@22\.1\.8\.1, graph\)' build.log \ + || fail "the report does not name llvm.libcxx as the C++ layer (graph)" build.log +ninja=$(ls target/*/*/build.ninja | head -1) +grep -q -- '-isystem[^ ]*xim-x-llvm[^ ]*include/c++/v1' "$ninja" \ + && fail "the payload's libc++ headers are still on a compile line" "$ninja" +grep -q -- '-nostdinc++' "$ninja" || fail "the compile lines carry no -nostdinc++" "$ninja" +grep -q -- '-nostdlib++' "$ninja" || fail "the link line carries no -nostdlib++" "$ninja" +grep -q -- 'xim-x-llvm[^ ]*libc++\(abi\)\?\.a' "$ninja" \ + && fail "the link line still names the payload's libc++ archives" "$ninja" +bin=$(ls target/*/*/bin/app | head -1) +ldd "$bin" | grep -q 'libc++' && fail "the artefact still links a libc++ shared object" <(ldd "$bin") +out=$("$bin") || fail "the program exited non-zero" +[ "$out" = "1-2-3" ] || fail "expected 1-2-3, got: $out" +echo "ok: llvm.libcxx supplies the C++ layer over the payload's C library, and the program runs" diff --git a/tests/e2e/664_a_universal_apk_is_two_legs_in_one_tree.sh b/tests/e2e/664_a_universal_apk_is_two_legs_in_one_tree.sh new file mode 100755 index 000000000..af5f92cf6 --- /dev/null +++ b/tests/e2e/664_a_universal_apk_is_two_legs_in_one_tree.sh @@ -0,0 +1,110 @@ +#!/usr/bin/env bash +# requires: elf gcc android-ndk +# 664_a_universal_apk_is_two_legs_in_one_tree.sh -- #630 A9. The route is +# chosen by the artifact's FORM, not by the target's kind: a `kind = "app"` +# target whose link form is a shared object on every requested row (every +# Android row) accepts several `--target` triples, the way a library target +# already does (`build_and_pack_library`). Each leg is staged as +# `lib//lib.so` into ONE tree; the declared deploy files are +# staged once; one `mcpp pack` invocation, one staged tree. +# +# `--format dir` is the built-in format that hands the staged tree over +# without compressing it and without needing a `mcpp:plugins` member — see +# `pack::run`'s `Format::Tar` check in `run_shared_program`, which is the +# only place staging and format interact for a shared program. +set -e + +t=$(mktemp -d); trap 'rm -rf "$t"' EXIT + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +cd "$t" +mkdir -p src +cat > mcpp.toml <<'TOML' +[package] +name = "myapp" +version = "0.1.0" + +[targets.myapp] +kind = "app" +main = "src/main.cpp" + +[[runtime.deploy]] +from = "res.txt" +to = "myres" +TOML +cat > src/main.cpp <<'CPP' +int main() { return 0; } +CPP +printf "resource-1\n" > res.txt + +# ── 1. one triple: the staged tree is unchanged (flat lib/.so) ────── +# +# This is the byte-identical direction (§ record item A9, "Do" #2): the +# multi-leg branch in `cmd_pack` is only reached for `triples.size() > 1`, so +# a single `--target` never enters it, and `Plan::extraSharedLegs` stays +# empty. Also asserted by 652b for the tarball route; repeated here, against +# `--format dir`, because THIS is the format the several-triple case below +# also uses, and the two must be compared under the same format. +"$MCPP" pack myapp --target aarch64-linux-android --format dir > pack1.log 2>&1 \ + || fail "single-target Android pack failed" pack1.log +staged1=$(ls -d target/dist/myapp-0.1.0-*/ 2>/dev/null | head -1) +[ -n "$staged1" ] || fail "no staged tree for the single-target pack" pack1.log +[ -f "${staged1}lib/libmyapp.so" ] \ + || fail "single-target pack: lib/libmyapp.so (flat) is missing" pack1.log +[ ! -d "${staged1}lib/arm64-v8a" ] \ + || fail "single-target pack introduced lib/arm64-v8a/ -- this must not change" pack1.log +[ -f "${staged1}bin/myres/res.txt" ] \ + || fail "single-target pack: the deploy'd file is missing" pack1.log +echo "one --target stages flat lib/libmyapp.so, unchanged OK" + +# ── 2. two Android triples: one tree, two lib// legs ───────────────── +rm -rf target/dist +"$MCPP" pack myapp --target aarch64-linux-android --target x86_64-linux-android \ + --format dir > pack2.log 2>&1 \ + || fail "two-target Android pack failed" pack2.log +staged2=$(ls -d target/dist/myapp-0.1.0-*/ 2>/dev/null | head -1) +[ -n "$staged2" ] || fail "no staged tree for the two-target pack" pack2.log + +[ -f "${staged2}lib/arm64-v8a/libmyapp.so" ] \ + || fail "lib/arm64-v8a/libmyapp.so is missing" pack2.log +file "${staged2}lib/arm64-v8a/libmyapp.so" | grep -qi "ARM aarch64" \ + || fail "lib/arm64-v8a/libmyapp.so is not an aarch64 object" pack2.log + +[ -f "${staged2}lib/x86_64/libmyapp.so" ] \ + || fail "lib/x86_64/libmyapp.so is missing" pack2.log +file "${staged2}lib/x86_64/libmyapp.so" | grep -qi "x86-64" \ + || fail "lib/x86_64/libmyapp.so is not an x86_64 object" pack2.log + +[ ! -f "${staged2}lib/libmyapp.so" ] \ + || fail "the flat lib/libmyapp.so also exists -- the multi-leg tree must not carry it" pack2.log +echo "two --target flags stage lib/arm64-v8a/ and lib/x86_64/ in one tree OK" + +# The declared deploy file travels ONCE, not once per leg. +depcount=$(find "${staged2}bin" -name res.txt | wc -l) +[ -f "${staged2}bin/myres/res.txt" ] || fail "the deploy'd file is missing from the multi-leg tree" pack2.log +[ "$depcount" -eq 1 ] \ + || fail "the deploy'd file was staged $depcount times, not once" pack2.log +echo "the deploy'd file is staged once across both legs OK" + +# ── 3. negative direction: an executable-form row refuses a second triple ─ +# +# `x86_64-linux-gnu` is not Android: `application_form` answers `Executable` +# there, so `accepts_several_targets` must refuse this request exactly as it +# refused before #630 A9 existed. +out=$("$MCPP" pack myapp --target x86_64-linux-gnu --target x86_64-linux-gnu 2>&1) \ + && fail "packing an app for two executable-form triples must be refused" <(echo "$out") +expected="--target may be given once when packing a program: an application bundle wraps one executable, and one executable has one target." +grep -qF -- "$expected" <<<"$out" \ + || fail "the refusal message does not match today's wording" <(echo "$out") +echo "two executable-form triples are refused with today's message OK" + +# Mixed direction: one Android (shared-object) row and one host (executable) +# row is not "every row is a shared object" either. +out=$("$MCPP" pack myapp --target aarch64-linux-android --target x86_64-linux-gnu 2>&1) \ + && fail "mixing a shared-object row with an executable row must be refused" <(echo "$out") +grep -qF -- "$expected" <<<"$out" \ + || fail "the mixed-row refusal message does not match today's wording" <(echo "$out") +echo "a shared-object row mixed with an executable row is refused OK" + +echo "664: the universal APK is the library route applied to an app OK" diff --git a/tests/e2e/665_a_host_tool_follows_its_source.sh b/tests/e2e/665_a_host_tool_follows_its_source.sh new file mode 100755 index 000000000..68860ab25 --- /dev/null +++ b/tests/e2e/665_a_host_tool_follows_its_source.sh @@ -0,0 +1,137 @@ +#!/usr/bin/env bash +# requires: gcc +# 665 -- a cached host tool follows its source (#630, item 6). The tool store +# was keyed on the package's version, so a change to a `path` tool's source +# under an unchanged version was invisible to every consumer until the version +# moved; the build reported success over the previous compiler's output. The +# key now carries a stamp of the tree for a `path` package and the resolved +# commit for a `git` package. Measured in both directions, at one path and +# with different bytes: an edit reaches the consumer, its reversal reaches it +# too, and a tree that did not change is not rebuilt (e2e 187's claim, which +# this test keeps). +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +cd "$TMP" + +# An isolated store, so that an entry from a previous run cannot decide. +export MCPP_HOME="$TMP/mcpphome" +mkdir -p "$MCPP_HOME" +if [ -d "$HOME/.mcpp/registry" ]; then + ln -s "$HOME/.mcpp/registry" "$MCPP_HOME/registry" +fi + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +write_tool() { # $1 = directory, $2 = the answer the generator writes + mkdir -p "$1/src" + cat > "$1/mcpp.toml" <<'EOF' +[package] +name = "toolpkg" +version = "0.1.0" + +[targets.gen] +kind = "bin" +main = "src/gen.cpp" +EOF + cat > "$1/src/gen.cpp" < +int main(int argc, char** argv) { + if (argc < 2) return 2; + FILE* f = std::fopen(argv[1], "w"); + if (!f) return 3; + std::fprintf(f, "int generated_answer() { return $2; }\n"); + std::fclose(f); + return 0; +} +EOF +} + +write_app() { # $1 = directory, $2 = the dependency table line + mkdir -p "$1/src" + cat > "$1/mcpp.toml" < "$1/src/main.cpp" <<'EOF' +#include +int generated_answer(); +int main() { std::printf("ANSWER=%d\n", generated_answer()); } +EOF + cat > "$1/build.mcpp" <<'EOF' +#include +#include +#include +import mcpp; +int main() { + const char* tool = mcpp::dep_bin("toolpkg", "gen"); + if (!tool || !*tool) { std::fprintf(stderr, "no tool path\n"); return 1; } + std::string out = std::string(mcpp::out_dir()) + "/gen.cpp"; + std::string cmd = std::string("\"") + tool + "\" \"" + out + "\""; + if (std::system(cmd.c_str()) != 0) { std::fprintf(stderr, "tool failed\n"); return 1; } + mcpp::generated(out.c_str()); +} +EOF +} + +answer() { # builds in the current directory and prints the program's answer + rm -rf target + "$MCPP" build > "$1" 2>&1 || fail "build failed" "$1" + "$MCPP" run 2>&1 | grep '^ANSWER=' | tail -1 +} + +# ── A `path` tool: same version, different bytes, both directions ──────────── +write_tool toolpkg 41 +write_app app 'toolpkg = { path = "../toolpkg", tools = ["gen"] }' +cd app +[ "$(answer b1.log)" = "ANSWER=41" ] || fail "first build: expected 41" b1.log +grep -q "Building.*host tool" b1.log || fail "the first build did not build the tool" b1.log + +# Unchanged tree: a store hit, no rebuild. (The rebuild line is the claim +# e2e 187 makes; a stamp that moved without a change would break it.) +sleep 1 +[ "$(answer b2.log)" = "ANSWER=41" ] || fail "second build: expected 41" b2.log +grep -q "Building.*host tool" b2.log && fail "an unchanged path tool was rebuilt" b2.log +echo "ok: an unchanged path tool is a store hit" + +# The edit, with the version untouched. +sleep 1 +write_tool ../toolpkg 42 +[ "$(answer b3.log)" = "ANSWER=42" ] || fail "after the edit: expected 42" b3.log +grep -q "Building.*host tool" b3.log || fail "the edit did not rebuild the tool" b3.log +echo "ok: an edit to a path tool reaches the consumer without a version bump" + +# And its reversal, which a key that only ever grew would miss. +sleep 1 +write_tool ../toolpkg 41 +[ "$(answer b4.log)" = "ANSWER=41" ] || fail "after the reversal: expected 41" b4.log +grep -q "Building.*host tool" b4.log || fail "the reversal did not rebuild the tool" b4.log +echo "ok: the reversal reaches the consumer too" + +# ── A `git` tool: keyed by the resolved commit ────────────────────────────── +cd "$TMP" +write_tool gittool 7 +git -C gittool init -q -b main +git -C gittool -c user.name=t -c user.email=t@t add -A +git -C gittool -c user.name=t -c user.email=t@t commit -qm "seven" +A=$(git -C gittool rev-parse HEAD) +write_tool gittool 8 +git -C gittool -c user.name=t -c user.email=t@t commit -qam "eight" +B=$(git -C gittool rev-parse HEAD) + +write_app gapp "toolpkg = { git = \"file://$TMP/gittool\", rev = \"$A\", tools = [\"gen\"] }" +cd gapp +[ "$(answer g1.log)" = "ANSWER=7" ] || fail "git rev A: expected 7" g1.log +sed -i "s/$A/$B/" mcpp.toml +[ "$(answer g2.log)" = "ANSWER=8" ] || fail "git rev B: expected 8" g2.log +grep -q "Building.*host tool" g2.log || fail "moving the pin to B did not rebuild the tool" g2.log +# Back to A: the entry built for A is still valid, so nothing is rebuilt. +sed -i "s/$B/$A/" mcpp.toml +[ "$(answer g3.log)" = "ANSWER=7" ] || fail "git rev A again: expected 7" g3.log +grep -q "Building.*host tool" g3.log && fail "the entry for commit A was not reused" g3.log +echo "ok: a git tool is keyed by its commit, and each commit's entry is reused" diff --git a/tests/e2e/666_a_macho_program_is_staged_without_its_closure.sh b/tests/e2e/666_a_macho_program_is_staged_without_its_closure.sh new file mode 100755 index 000000000..bcb59dddb --- /dev/null +++ b/tests/e2e/666_a_macho_program_is_staged_without_its_closure.sh @@ -0,0 +1,80 @@ +#!/usr/bin/env bash +# requires: macos +# 666 -- on the one host that produces a Mach-O program, `mcpp pack` stages +# the program and its declared files and hands the tree to a dispatched +# format with `closure = not-walked` and a reason (#630, item 3a). Before +# this, the Mach-O refusal preceded staging, and a bundler reached its action +# with no tree at all. The negative direction stays: `--format dir` and +# `--format tar`, whose product is the closure, still refuse. +set -e + +TMP=$(mktemp -d) +trap "rm -rf $TMP" EXIT +export MCPP_HOME=$HOME/.mcpp + +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } + +cd "$TMP" +"$MCPP" new zapapp > /dev/null +cd zapapp +mkdir -p share +printf 'notes\n' > share/notes.txt +cat >> mcpp.toml <<'EOF' + +[runtime] +deploy = [ { from = "share/notes.txt", to = "data" } ] +EOF +cat > dist.sh <<'EOF' +set -e +stage="$1"; manifest="$2"; out="$3" +{ + echo "staged:" + ( cd "$stage" && find . -type f | sort ) + echo "manifest:" + sed -n '1,2p' "$manifest" +} > "$out" +EOF +chmod +x dist.sh +cat > build.mcpp <<'EOF' +import std; +import mcpp; +int main() { + mcpp::provides_pack_format("zap"); + if (std::string_view(mcpp::pack_format()) != "zap") return 0; + const std::string root = mcpp::manifest_dir(); + const std::string stage = std::string("${mcpp.stage_dir}"); + const std::string out = std::string(mcpp::out_dir()) + "/app.zap"; + mcpp::action a; + a.id = "zap"; + a.role = "artifact"; + a.description = "zap"; + a.arg((root + "/dist.sh").c_str()) + .arg(stage.c_str()) + .arg((stage + ".stage-manifest").c_str()) + .arg(out.c_str()) + .input("${mcpp.target_file:zapapp}") + .output(out.c_str()) + .submit(); + return 0; +} +EOF + +# The negative direction first: the built-in archive still refuses, since +# its product IS the closure. +if "$MCPP" pack --format dir > dir.log 2>&1; then + fail "--format dir of a Mach-O program was accepted" dir.log +fi +grep -q 'Mach-O' dir.log || fail "the refusal does not name the format" dir.log +echo " negative: --format dir still refuses a Mach-O program" + +"$MCPP" pack --format zap > zap.log 2>&1 || fail "mcpp pack --format zap failed" zap.log +grep -q 'staged without its dependency closure' zap.log \ + || fail "pack did not report the tree as staged without its closure" zap.log +Z=$(find target -name 'app.zap' | head -1) +[ -n "$Z" ] || fail "--format zap produced nothing" zap.log +grep -qx './bin/zapapp' "$Z" || fail "the action's view of the tree lacks bin/zapapp" "$Z" zap.log +grep -qx './bin/data/notes.txt' "$Z" || fail "the action's view of the tree lacks the deployed file" "$Z" zap.log +grep -qx 'closure = not-walked' "$Z" || fail "the manifest does not say closure = not-walked" "$Z" zap.log +grep -q '^reason = ' "$Z" || fail "the manifest carries no reason line" "$Z" zap.log +echo " positive: the dispatched format sees the program, the deployed file, and closure = not-walked" +echo "PASS: 666_a_macho_program_is_staged_without_its_closure" diff --git a/tests/unit/test_artifact_naming.cpp b/tests/unit/test_artifact_naming.cpp index c49776ea5..fa8b24c63 100644 --- a/tests/unit/test_artifact_naming.cpp +++ b/tests/unit/test_artifact_naming.cpp @@ -204,6 +204,20 @@ TEST(ArtifactNaming, ApplicationFormOnTheHostTripleIsExecutable) { EXPECT_EQ(tr::application_form(tr::Triple{}), tr::ApplicationForm::Executable); } +// ── #630 A9: the Android ABI name of a row's architecture ─────────────────── +// +// The two rows mcpp currently produces a shared object for; a third +// architecture keeps its GNU spelling rather than a guessed Android name. + +TEST(AndroidAbi, TranslatesTheTwoKnownArchitectures) { + EXPECT_EQ(tr::android_abi(T("aarch64-linux-android")), "arm64-v8a"); + EXPECT_EQ(tr::android_abi(T("x86_64-linux-android")), "x86_64"); +} + +TEST(AndroidAbi, AnUntranslatedArchitectureKeepsItsGnuSpelling) { + EXPECT_EQ(tr::android_abi(T("riscv64-linux-android")), "riscv64"); +} + } // namespace // `platform_name` (#622 A7): the triple's `os`, except that an `env` naming a diff --git a/tests/unit/test_cfg_os_only_platform.cpp b/tests/unit/test_cfg_os_only_platform.cpp new file mode 100644 index 000000000..7f1f0ea65 --- /dev/null +++ b/tests/unit/test_cfg_os_only_platform.cpp @@ -0,0 +1,135 @@ +#include + +import std; +import mcpp.build.prepare_inputs; + +// #630 item 7 — "a selector that names only an operating system is a +// platform" (design record 2026-09-13-630 §8.2). `os_only_platforms` is the +// function `mcpp emit xpkg` (`mcpp::pm::emit_xpkg`) asks to decide whether a +// `[target.]` conditional tool declaration can be folded into one +// of the descriptor's three platform blocks (`linux`, `macosx`, `windows`) +// instead of only producing the `publish/target-axis-tools` advisory. +// +// Both directions are asserted: the positive claim (the seven forms §8.2 +// lists each map onto the block(s) they name) and the negative claim (a +// predicate mentioning anything else — an architecture, an environment, a +// combinator, `not(...)`, even one built entirely from OS terms — is refused, +// which is what keeps the descriptor from claiming an install-time edge on a +// platform the predicate does not unconditionally name). + +namespace cfgpred = mcpp::build::cfgpred; + +namespace { +bool has(const std::vector& v, std::string_view a) { + return std::ranges::find(v, a) != v.end(); +} +} // namespace + +// ── Positive: the seven OS-only forms ─────────────────────────────────────── + +TEST(OsOnlyPlatforms, BareAliasWindows) { + auto p = cfgpred::os_only_platforms("windows"); + EXPECT_EQ(p, std::vector({"windows"})); +} + +TEST(OsOnlyPlatforms, CfgWindows) { + auto p = cfgpred::os_only_platforms("cfg(windows)"); + EXPECT_EQ(p, std::vector({"windows"})); +} + +TEST(OsOnlyPlatforms, CfgOsEqualsLinux) { + auto p = cfgpred::os_only_platforms(R"(cfg(os = "linux"))"); + EXPECT_EQ(p, std::vector({"linux"})); +} + +TEST(OsOnlyPlatforms, CfgLinux) { + auto p = cfgpred::os_only_platforms("cfg(linux)"); + EXPECT_EQ(p, std::vector({"linux"})); +} + +TEST(OsOnlyPlatforms, CfgMacos) { + auto p = cfgpred::os_only_platforms("cfg(macos)"); + // The descriptor's spelling is `macosx`, not `macos` — see + // `mcpp::pm::emit_xpkg`'s three blocks. + EXPECT_EQ(p, std::vector({"macosx"})); +} + +TEST(OsOnlyPlatforms, CfgOsEqualsWindows) { + auto p = cfgpred::os_only_platforms(R"(cfg(os = "windows"))"); + EXPECT_EQ(p, std::vector({"windows"})); +} + +TEST(OsOnlyPlatforms, CfgOsEqualsMacos) { + auto p = cfgpred::os_only_platforms(R"(cfg(os = "macos"))"); + EXPECT_EQ(p, std::vector({"macosx"})); +} + +TEST(OsOnlyPlatforms, CfgUnixNamesBothMacAndLinux) { + // `unix` means `family == "unix"`, which macOS and Linux both satisfy — + // the only one of the seven forms that names two blocks at once. + auto p = cfgpred::os_only_platforms("cfg(unix)"); + EXPECT_EQ(p.size(), 2u); + EXPECT_TRUE(has(p, "linux")); + EXPECT_TRUE(has(p, "macosx")); + EXPECT_FALSE(has(p, "windows")); +} + +TEST(OsOnlyPlatforms, BareAliasUnix) { + auto p = cfgpred::os_only_platforms("unix"); + EXPECT_EQ(p.size(), 2u); + EXPECT_TRUE(has(p, "linux")); + EXPECT_TRUE(has(p, "macosx")); +} + +// ── Negative: everything the record says keeps the warning ───────────────── + +TEST(OsOnlyPlatforms, ArchitectureIsNotOsOnly) { + auto p = cfgpred::os_only_platforms(R"(cfg(target_arch = "aarch64"))"); + EXPECT_TRUE(p.empty()); +} + +TEST(OsOnlyPlatforms, CombinatorOfOsAndArchIsNotOsOnly) { + auto p = cfgpred::os_only_platforms( + R"(cfg(all(linux, target_arch = "x86_64")))"); + EXPECT_TRUE(p.empty()); +} + +TEST(OsOnlyPlatforms, NotWindowsIsNotOsOnly) { + // Syntactically this names only an OS, but `not(...)` answers an + // unbounded set of platforms (everything this vocabulary does not yet + // call "windows"), not the two blocks "linux and macosx" would be if this + // were treated as their union. Kept conservative: any combinator + // disqualifies, `not` included. + auto p = cfgpred::os_only_platforms("cfg(not(windows))"); + EXPECT_TRUE(p.empty()); +} + +TEST(OsOnlyPlatforms, EnvIsNotOsOnly) { + auto p = cfgpred::os_only_platforms(R"(cfg(env = "android"))"); + EXPECT_TRUE(p.empty()); +} + +TEST(OsOnlyPlatforms, LayerKeyIsNotOsOnly) { + auto p = cfgpred::os_only_platforms(R"(cfg(compiler = "llvm"))"); + EXPECT_TRUE(p.empty()); +} + +TEST(OsOnlyPlatforms, AnAllOsCombinatorIsStillNotOsOnly) { + // Even a combinator built ENTIRELY from OS terms is refused — see the + // comment on `os_only_platforms` for why `any(linux, macos)` is not the + // same claim as either block alone. + auto p = cfgpred::os_only_platforms("cfg(any(linux, macos))"); + EXPECT_TRUE(p.empty()); +} + +TEST(OsOnlyPlatforms, ExactTripleIsNotOsOnly) { + // An exact triple shares no grammar with cfg()/the bare aliases; it names + // one target, not a platform. + auto p = cfgpred::os_only_platforms("x86_64-linux-musl"); + EXPECT_TRUE(p.empty()); +} + +TEST(OsOnlyPlatforms, UnrecognisedOsValueIsNotOsOnly) { + auto p = cfgpred::os_only_platforms(R"(cfg(os = "freebsd"))"); + EXPECT_TRUE(p.empty()); +} diff --git a/tests/unit/test_hostflags.cpp b/tests/unit/test_hostflags.cpp index b9535c5d7..ceb56a1f9 100644 --- a/tests/unit/test_hostflags.cpp +++ b/tests/unit/test_hostflags.cpp @@ -359,11 +359,15 @@ TEST(HostFlags, DeploymentTargetOnlyOnMacos) { namespace { +// The openkal shape: the C library is the graph's too. `cAbiPrebuilt` is set +// explicitly because it decides the Mach-O emulated-TLS row below, and the +// default (`true`) describes the other arrangement. mcpp::toolchain::Toolchain graph_tc(std::string triple) { mcpp::toolchain::Toolchain tc; tc.compiler = CompilerId::Clang; tc.targetTriple = std::move(triple); tc.targetCxxRuntime = true; + tc.cAbiPrebuilt = false; return tc; } @@ -410,6 +414,25 @@ TEST(GraphRuntimeFlags, IosTakesTheSameFlagsAsMacOS) { EXPECT_TRUE(has(f, "-fvisibility-inlines-hidden")); } +// A hosted Mach-O target whose C library is a located SDK (the iOS rows over +// `llvm.libcxx`, mcpp#630) has dyld and the native TLS model with it, so the +// emulated-TLS row does not apply; the visibility rows still do, since a +// second libc++ in one process is kept apart by visibility on Mach-O. +TEST(GraphRuntimeFlags, MachOOverAPrebuiltCLibraryKeepsNativeTls) { + auto tc = graph_tc("aarch64-ios"); + tc.cAbiPrebuilt = true; + auto f = mcpp::toolchain::graph_runtime_compile_flags(tc); + EXPECT_FALSE(has(f, "-femulated-tls")); + EXPECT_FALSE(has(f, "-fdwarf-exceptions")); + EXPECT_TRUE(has(f, "-fvisibility=hidden")); + EXPECT_TRUE(has(f, "-fvisibility-inlines-hidden")); + // And PE keeps it regardless: `_tls_index` is loader-bootstrapped there + // whether or not the C library is the graph's. + auto pe = graph_tc("x86_64-windows-gnu"); + pe.cAbiPrebuilt = true; + EXPECT_TRUE(has(mcpp::toolchain::graph_runtime_compile_flags(pe), "-femulated-tls")); +} + // ELF takes NONE of them, and that is a decision rather than an omission. // There a `thread_local` is a fixed offset from the thread pointer, which the // C library establishes itself; adding the flag would work, cost an @@ -617,3 +640,86 @@ TEST(HostFlags, AnOwnSysrootTargetIsToldWhichTargetAndNothingElse) { EXPECT_TRUE(mcpp::toolchain::host_compile_tokens( bare, opt, mcpp::toolchain::no_escape).empty()); } + +// ── The C++ headers are the C++ layer's question ───────────────────────────── +// +// The payload's libc++ `-isystem` block was withheld exactly when the C +// LIBRARY came from the graph, which was the same question while the only +// graph C++ runtime sat over a graph C library. A hosted target whose C +// library is a located SDK while a package supplies libc++ (the iOS rows over +// `llvm.libcxx`, mcpp#630) answered "payload" there and put two libc++ header +// sets on one command line. The same fixture as the bypass test above: two +// empty files beside an `include/c++/v1` are a complete driver model. +TEST(HostFlags, TheCxxLayerDecidesWhoseLibcxxHeadersAreEmitted) { + namespace fs = std::filesystem; + const auto root = fs::temp_directory_path() / "mcpp_hostflags_cxx_fixture"; + fs::remove_all(root); + fs::create_directories(root / "bin"); + fs::create_directories(root / "include" / "c++" / "v1"); + { std::ofstream(root / "bin" / "clang++"); } + { std::ofstream(root / "bin" / "clang++.cfg"); } + struct Cleanup { + fs::path p; + ~Cleanup() { std::error_code ec; fs::remove_all(p, ec); } + } cleanup{root}; + + auto tc = tc_for(CompilerId::Clang); + tc.binaryPath = root / "bin" / "clang++"; + ASSERT_TRUE(mcpp::toolchain::resolve_clang_driver(tc).hasCfg) + << "the fixture did not produce a cfg — the assertions below would be vacuous"; + + const auto any_payload_cxx = [&](const std::vector& v) { + return std::ranges::any_of(v, [&](const std::string& t) { + return t.starts_with("-isystem") + && t.find((root / "include" / "c++" / "v1").string()) != std::string::npos; + }); + }; + const auto has = [](const std::vector& v, std::string_view f) { + return std::ranges::find(v, f) != v.end(); + }; + + HostFlagOptions payload; + payload.cfgBypass = HostFlagOptions::CfgBypass::Always; + payload.cAbiPrebuilt = true; + + // The baseline: a prebuilt C library under the payload's own libc++ takes + // the payload's headers, as every native build does. + const auto a = mcpp::toolchain::host_compile_tokens(tc, payload, mcpp::toolchain::no_escape); + EXPECT_TRUE(any_payload_cxx(a)); + EXPECT_TRUE(has(a, "-nostdinc++")); + + // A graph C++ runtime over the same prebuilt C library: the payload's + // headers are withheld and the driver's own search is closed, since the + // package's directories arrive through the target-side broadcast. + HostFlagOptions graph = payload; + graph.cxxFromGraph = true; + const auto b = mcpp::toolchain::host_compile_tokens(tc, graph, mcpp::toolchain::no_escape); + EXPECT_FALSE(any_payload_cxx(b)); + EXPECT_TRUE(has(b, "-nostdinc++")); + EXPECT_TRUE(has(b, "--no-default-config")); + + // An Apple cross target whose runtime is the SDK's libc++ and whose graph + // does not import `std`: prepare chose the SDK's headers, named + // explicitly because clang's Darwin driver would otherwise prefer the + // copy beside itself. + HostFlagOptions sdk = payload; + sdk.appleSdkRoot = fs::path("/Sdk/iPhoneSimulator.sdk"); + sdk.appleSdkCxxHeaders = true; + const auto c = mcpp::toolchain::host_compile_tokens(tc, sdk, mcpp::toolchain::no_escape); + EXPECT_FALSE(any_payload_cxx(c)); + EXPECT_TRUE(has(c, "-nostdinc++")); + EXPECT_TRUE(has(c, "-isystem" + (fs::path("/Sdk/iPhoneSimulator.sdk") / "usr" / "include" / "c++" / "v1").string())); + + // And with the graph runtime on that same target the SDK's headers are + // not named either: one libc++ per command line, whichever it is. + HostFlagOptions sdkGraph = sdk; + sdkGraph.cxxFromGraph = true; + sdkGraph.appleSdkCxxHeaders = false; + const auto d = mcpp::toolchain::host_compile_tokens(tc, sdkGraph, mcpp::toolchain::no_escape); + EXPECT_FALSE(any_payload_cxx(d)); + EXPECT_FALSE(std::ranges::any_of(d, [](const std::string& t) { + return t.starts_with("-isystem") && t.find("iPhoneSimulator.sdk") != std::string::npos; + })); + EXPECT_TRUE(has(d, "-nostdinc++")); +} + diff --git a/tests/unit/test_manifest.cpp b/tests/unit/test_manifest.cpp index 7bd3136ba..9b7d8c577 100644 --- a/tests/unit/test_manifest.cpp +++ b/tests/unit/test_manifest.cpp @@ -4668,8 +4668,12 @@ runnerX = ["qemu-aarch64-static"] ASSERT_TRUE(m.has_value()) << m.error().format(); ASSERT_EQ(m->schemaWarnings.size(), 1u); EXPECT_NE(m->schemaWarnings[0].find("'runnerX'"), std::string::npos) << m->schemaWarnings[0]; + // #610 / #630 item 5 — `min_api_level` joined this list; see + // tests/unit/test_target_scalar_keys.cpp for the test that keeps this + // string (and the array beside it) from drifting from the parser again. EXPECT_NE(m->schemaWarnings[0].find( - "Supported keys: cxx_runtime, linkage, runner, sysroot, toolchain"), + "Supported keys: cxx_runtime, linkage, min_api_level, runner, " + "sysroot, toolchain"), std::string::npos) << m->schemaWarnings[0]; } diff --git a/tests/unit/test_pack_binfmt.cpp b/tests/unit/test_pack_binfmt.cpp index 6f84f3bb6..175035b1d 100644 --- a/tests/unit/test_pack_binfmt.cpp +++ b/tests/unit/test_pack_binfmt.cpp @@ -28,6 +28,15 @@ void put(std::string& b, std::size_t at, std::uint64_t v, std::size_t width) { b[at + i] = static_cast((v >> (8 * i)) & 0xFF); } +// Big-endian counterpart, for the Mach-O fixtures below: a FAT header is +// always big-endian on disk, and one thin fixture is deliberately built with +// a byte-swapped (CIGAM) magic to exercise that leg of the reader. +void put_be(std::string& b, std::size_t at, std::uint64_t v, std::size_t width) { + if (b.size() < at + width) b.resize(at + width, '\0'); + for (std::size_t i = 0; i < width; ++i) + b[at + width - 1 - i] = static_cast((v >> (8 * i)) & 0xFF); +} + std::filesystem::path write_temp(std::string_view tag, std::string_view bytes) { auto p = std::filesystem::temp_directory_path() / std::format("mcpp-binfmt-{}-{}", tag, @@ -236,6 +245,110 @@ struct TempFile { TempFile& operator=(const TempFile&) = delete; }; +// ─── minimal Mach-O objects (thin and fat) ─────────────────────────────── +// +// Same reasoning as the ELF and PE fixtures above: a real Mach-O needs a +// macOS toolchain to produce, which is the dependency this reader exists to +// remove. Hand-built headers make "found these two names at these load +// commands, in this order" exact. + +// mach/machine.h. Kept local to the fixture builder rather than imported +// from `mcpp.pack.binfmt`, which does not export them (a fixture is allowed +// to know the format's own constants; a caller of the module should not have +// to). +constexpr std::uint32_t kCpuTypeX86_64 = 0x01000007; +constexpr std::uint32_t kCpuTypeArm64 = 0x0100000c; + +constexpr std::uint32_t kLcLoadDylib = 0x0000000c; +constexpr std::uint32_t kLcLoadWeakDylib = 0x80000018; +constexpr std::uint32_t kLcRpath = 0x8000001c; + +// One load command to bake into a fixture: `cmd` plus the string it carries. +// `structSize` is 24 for a dylib_command (name is the first field, after +// timestamp/current_version/compatibility_version at offsets 12/16/20 sit +// before the string) and 12 for an rpath_command (path is the only field +// after cmd/cmdsize). +struct Lc { std::uint32_t cmd; std::size_t structSize; std::string str; }; + +constexpr std::size_t kDylibStructSize = 24; +constexpr std::size_t kRpathStructSize = 12; + +// A thin Mach-O object: `is64` selects `mach_header`/`mach_header_64`, +// `bigEndian` selects the plain or the CIGAM (byte-swapped) magic, and every +// integer in the header and its load commands is written in the byte order +// `bigEndian` names — exactly what a real byte-swapped object would contain, +// and what `macho_thin_needed` has to undo to read it. +std::string macho_thin(bool is64, bool bigEndian, std::uint32_t cputype, + const std::vector& cmds) +{ + auto putN = [&](std::string& b, std::size_t at, std::uint64_t v, std::size_t w) { + if (bigEndian) put_be(b, at, v, w); else put(b, at, v, w); + }; + + std::string magic; + if (is64) magic = bigEndian ? std::string("\xfe\xed\xfa\xcf", 4) + : std::string("\xcf\xfa\xed\xfe", 4); + else magic = bigEndian ? std::string("\xfe\xed\xfa\xce", 4) + : std::string("\xce\xfa\xed\xfe", 4); + + const std::size_t headerSize = is64 ? 32 : 28; + std::string b(headerSize, '\0'); + std::copy(magic.begin(), magic.end(), b.begin()); + putN(b, 4, cputype, 4); // cputype + putN(b, 8, 0, 4); // cpusubtype + putN(b, 12, 2, 4); // filetype = MH_EXECUTE + putN(b, 16, cmds.size(), 4); // ncmds + putN(b, 20, 0, 4); // sizeofcmds, filled in below + putN(b, 24, 0, 4); // flags + if (is64) putN(b, 28, 0, 4); // reserved + + std::size_t cursor = headerSize; + for (auto const& lc : cmds) { + const std::size_t rawSize = lc.structSize + lc.str.size() + 1; + // Padded to a 4-byte boundary, as a real linker's cmdsize is — + // exercising that the reader trusts `cmdsize` to advance, not the + // string's own length. + const std::size_t cmdsize = (rawSize + 3) & ~std::size_t(3); + b.resize(cursor + cmdsize, '\0'); + putN(b, cursor + 0, lc.cmd, 4); + putN(b, cursor + 4, cmdsize, 4); + putN(b, cursor + 8, lc.structSize, 4); // lc_str offset from cmd start + std::copy(lc.str.begin(), lc.str.end(), b.begin() + static_cast(cursor + lc.structSize)); + cursor += cmdsize; + } + putN(b, 20, cursor - headerSize, 4); // sizeofcmds, now that it is known + return b; +} + +// A fat (universal) Mach-O: FAT_MAGIC plus one 20-byte `fat_arch` entry per +// slice, ALWAYS big-endian regardless of what the slices themselves are. +std::string macho_fat(const std::vector>& slices) +{ + const std::size_t headerSize = 8 + slices.size() * 20; + std::vector sliceOffsets; + std::size_t cursor = headerSize; + for (auto const& [cputype, bytes] : slices) { + sliceOffsets.push_back(cursor); + cursor += bytes.size(); + } + + std::string b(cursor, '\0'); + put_be(b, 0, 0xcafebabe, 4); // FAT_MAGIC + put_be(b, 4, slices.size(), 4); // nfat_arch + for (std::size_t i = 0; i < slices.size(); ++i) { + const std::size_t at = 8 + i * 20; + put_be(b, at + 0, slices[i].first, 4); // cputype + put_be(b, at + 4, 0, 4); // cpusubtype + put_be(b, at + 8, sliceOffsets[i], 4); // offset + put_be(b, at + 12, slices[i].second.size(), 4); // size + put_be(b, at + 16, 0, 4); // align + } + for (std::size_t i = 0; i < slices.size(); ++i) + std::copy(slices[i].second.begin(), slices[i].second.end(), + b.begin() + static_cast(sliceOffsets[i])); + return b; +} + } // namespace TEST(PackBinfmt, IdentifiesElfWithoutRunningIt) { @@ -377,6 +490,176 @@ TEST(PackBinfmt, TheSystemPredicateKnowsWindowsFromTheToolset) { EXPECT_FALSE(bf::is_system_lib(bf::Format::Elf, "libcurl.so.4")); } +// ─── Mach-O: #630 §4 — the reader `needed_names` used to refuse ───────── + +TEST(PackBinfmt, IdentifiesThinAndFatMachOWithoutRunningIt) { + TempFile thin{"macho-thin", + macho_thin(/*is64=*/true, /*bigEndian=*/false, kCpuTypeArm64, + {{kLcLoadDylib, kDylibStructSize, "/usr/lib/libSystem.B.dylib"}})}; + EXPECT_EQ(bf::identify(thin.path).format, bf::Format::MachO); + + TempFile fat{"macho-fat", + macho_fat({{kCpuTypeX86_64, macho_thin(true, false, kCpuTypeX86_64, {})}, + {kCpuTypeArm64, macho_thin(true, false, kCpuTypeArm64, {})}})}; + EXPECT_EQ(bf::identify(fat.path).format, bf::Format::MachO); +} + +TEST(PackBinfmt, ThinMachOReadsNamesAndRpathsInLoadCommandOrder) { + // Two LC_LOAD_DYLIB, one LC_LOAD_WEAK_DYLIB, two LC_RPATH — the record's + // §4.3 fixture shape, in one file. + TempFile f{"macho-thin-arm64", + macho_thin(true, false, kCpuTypeArm64, { + {kLcLoadDylib, kDylibStructSize, "@rpath/libfoo.dylib"}, + {kLcLoadDylib, kDylibStructSize, "/usr/lib/libSystem.B.dylib"}, + {kLcLoadWeakDylib, kDylibStructSize, "@rpath/libbar.dylib"}, + {kLcRpath, kRpathStructSize, "@executable_path/../Frameworks"}, + {kLcRpath, kRpathStructSize, "@loader_path/../lib"}, + })}; + + auto r = bf::macho_needed(f.path, "aarch64"); + ASSERT_TRUE(r.has_value()) << r.error(); + EXPECT_EQ(r->names, (std::vector{ + "@rpath/libfoo.dylib", "/usr/lib/libSystem.B.dylib", "@rpath/libbar.dylib"})); + EXPECT_EQ(r->rpaths, (std::vector{ + "@executable_path/../Frameworks", "@loader_path/../lib"})); + + // `needed_names` completes for Mach-O now: names alone, through the same + // dispatcher ELF and PE already go through. + auto names = bf::needed_names(f.path); + ASSERT_TRUE(names.has_value()) << names.error(); + EXPECT_EQ(*names, r->names); +} + +TEST(PackBinfmt, AWrongArchRequestOnAThinFileStillReadsIt) { + // There is only one slice in a thin file; refusing to read it because + // the caller asked for a different architecture would be wrong — the + // caller already knows what it built, and mcpp never builds a thin + // Mach-O whose OWN cputype disagrees with the triple that produced it. + TempFile f{"macho-thin-wrongarch", + macho_thin(true, false, kCpuTypeArm64, + {{kLcLoadDylib, kDylibStructSize, "@rpath/libfoo.dylib"}})}; + auto r = bf::macho_needed(f.path, "x86_64"); + ASSERT_TRUE(r.has_value()) << r.error(); + EXPECT_EQ(r->names, (std::vector{"@rpath/libfoo.dylib"})); +} + +TEST(PackBinfmt, FatMachOReadsOnlyTheRequestedSlice) { + auto x64 = macho_thin(true, false, kCpuTypeX86_64, + {{kLcLoadDylib, kDylibStructSize, "onlyintel.dylib"}}); + auto arm = macho_thin(true, false, kCpuTypeArm64, + {{kLcLoadDylib, kDylibStructSize, "onlyarm.dylib"}}); + TempFile fat{"macho-fat-differing", + macho_fat({{kCpuTypeX86_64, x64}, {kCpuTypeArm64, arm}})}; + + auto arm64Read = bf::macho_needed(fat.path, "aarch64"); + ASSERT_TRUE(arm64Read.has_value()) << arm64Read.error(); + EXPECT_EQ(arm64Read->names, (std::vector{"onlyarm.dylib"})); + // A name present only in the OTHER slice must be absent — the whole + // point of selecting by architecture rather than reading both. + EXPECT_EQ(std::ranges::find(arm64Read->names, "onlyintel.dylib"), + arm64Read->names.end()); + + auto x64Read = bf::macho_needed(fat.path, "x86_64"); + ASSERT_TRUE(x64Read.has_value()) << x64Read.error(); + EXPECT_EQ(x64Read->names, (std::vector{"onlyintel.dylib"})); +} + +TEST(PackBinfmt, ABigEndianMagicThinMachOIsReadCorrectly) { + // The CIGAM leg: every integer in the header and its load commands is + // written big-endian, which `macho_thin_needed` has to detect from the + // magic alone and undo. + TempFile f{"macho-thin-be", + macho_thin(/*is64=*/true, /*bigEndian=*/true, kCpuTypeArm64, + {{kLcLoadDylib, kDylibStructSize, "/usr/lib/libSystem.B.dylib"}, + {kLcRpath, kRpathStructSize, "@executable_path/../lib"}})}; + auto r = bf::macho_needed(f.path, {}); + ASSERT_TRUE(r.has_value()) << r.error(); + EXPECT_EQ(r->names, (std::vector{"/usr/lib/libSystem.B.dylib"})); + EXPECT_EQ(r->rpaths, (std::vector{"@executable_path/../lib"})); +} + +TEST(PackBinfmt, TheSystemPredicateKnowsMachOsOwnRoots) { + EXPECT_TRUE(bf::is_system_lib(bf::Format::MachO, "/usr/lib/libSystem.B.dylib")); + EXPECT_TRUE(bf::is_system_lib(bf::Format::MachO, + "/System/Library/Frameworks/Foundation.framework/Foundation")); + // Not yet resolved to a file -- `@rpath` names a search, not a root. + EXPECT_FALSE(bf::is_system_lib(bf::Format::MachO, "@rpath/libc++.1.dylib")); +} + +TEST(PackBinfmt, ResolveMachoNamesTriesRpathsInOrderAndReportsUnresolved) { + auto dir = std::filesystem::temp_directory_path() + / std::format("mcpp-macho-resolve-{}", + std::chrono::steady_clock::now().time_since_epoch().count()); + auto bin = dir / "bin"; + auto first = dir / "first"; + auto second = dir / "second"; + std::filesystem::create_directories(bin); + std::filesystem::create_directories(first); + std::filesystem::create_directories(second); + // The SAME leaf name in both candidate directories, so a resolution that + // ignores rpath order cannot be told apart from one that respects it -- + // only the CONTENT of which file won can. + std::ofstream{first / "dup.dylib"} << "first"; + std::ofstream{second / "dup.dylib"} << "second"; + std::ofstream{first / "onlysecond-decoy.dylib"} << "unused"; + + std::vector names{ + "@rpath/dup.dylib", + "@rpath/nowhere.dylib", + }; + std::vector rpaths{ + "@executable_path/../first", + "@executable_path/../second", + }; + + auto resolved = bf::resolve_macho_names(names, rpaths, bin, bin); + ASSERT_EQ(resolved.size(), 2u); + + EXPECT_EQ(resolved[0].name, "@rpath/dup.dylib"); + EXPECT_FALSE(resolved[0].unresolved); + // `lexically_normal`: the resolver joins `@executable_path/../first` onto + // `bin` without collapsing the `..` itself (that is the filesystem's job, + // at `exists()`), so the raw and the hand-built path differ textually + // while naming the same file. + EXPECT_EQ(resolved[0].path.lexically_normal(), (first / "dup.dylib").lexically_normal()) + << "the FIRST rpath entry must win when both would resolve"; + + // Reported, not skipped: the vector still carries an entry for the name + // that resolved nowhere. + EXPECT_EQ(resolved[1].name, "@rpath/nowhere.dylib"); + EXPECT_TRUE(resolved[1].unresolved); + EXPECT_TRUE(resolved[1].path.empty()); + + std::error_code ec; + std::filesystem::remove_all(dir, ec); +} + +TEST(PackBinfmt, ResolveMachoNamesSubstitutesExecutableAndLoaderPath) { + auto dir = std::filesystem::temp_directory_path() + / std::format("mcpp-macho-resolve2-{}", + std::chrono::steady_clock::now().time_since_epoch().count()); + auto exeDir = dir / "bin"; + auto loaderDir = dir / "lib" / "plugins"; + std::filesystem::create_directories(exeDir); + std::filesystem::create_directories(loaderDir); + std::ofstream{exeDir / "libbeside.dylib"} << "x"; + std::ofstream{loaderDir / "libplugin.dylib"} << "x"; + + std::vector names{ + "@executable_path/libbeside.dylib", + "@loader_path/libplugin.dylib", + }; + auto resolved = bf::resolve_macho_names(names, {}, exeDir, loaderDir); + ASSERT_EQ(resolved.size(), 2u); + EXPECT_EQ(resolved[0].path, exeDir / "libbeside.dylib"); + EXPECT_FALSE(resolved[0].unresolved); + EXPECT_EQ(resolved[1].path, loaderDir / "libplugin.dylib"); + EXPECT_FALSE(resolved[1].unresolved); + + std::error_code ec; + std::filesystem::remove_all(dir, ec); +} + // ─── the zip writer ────────────────────────────────────────────────────── TEST(PackZip, Crc32MatchesTheKnownVectors) { diff --git a/tests/unit/test_pack_modes.cpp b/tests/unit/test_pack_modes.cpp index be88dfd7f..d63d72f79 100644 --- a/tests/unit/test_pack_modes.cpp +++ b/tests/unit/test_pack_modes.cpp @@ -6,6 +6,9 @@ import mcpp.pack; using mcpp::pack::Mode; using mcpp::pack::parse_mode; using mcpp::pack::mode_cli_name; +using mcpp::pack::Format; +using mcpp::pack::ClosureUnavailableOutcome; +using mcpp::pack::closure_unavailable_outcome; TEST(PackModes, CanonicalNamesParse) { EXPECT_EQ(parse_mode("system"), Mode::None); @@ -29,3 +32,23 @@ TEST(PackModes, CliNamesAreCanonical) { EXPECT_EQ(mode_cli_name(Mode::BundleAll), "self-contained"); EXPECT_EQ(mode_cli_name(Mode::Static), "static"); } + +// ── #630 §3: what happens when a format's dependency closure is unavailable, +// as a function of the format alone ────────────────────────────────────── +// +// The archive formats (`tar`, `dir`) ARE the closure -- an unavailable one is +// the command failing, exactly as before `pack::run` staged declared files +// ahead of the closure walk. A dispatched format receives the staged tree +// regardless: a provider may not need a closure at all. + +TEST(PackClosureUnavailableOutcome, ArchiveFormatsFailTheCommand) { + EXPECT_EQ(closure_unavailable_outcome(Format::Tar), + ClosureUnavailableOutcome::CommandFails); + EXPECT_EQ(closure_unavailable_outcome(Format::Dir), + ClosureUnavailableOutcome::CommandFails); +} + +TEST(PackClosureUnavailableOutcome, DispatchedFormatStagesWithoutIt) { + EXPECT_EQ(closure_unavailable_outcome(Format::Dispatched), + ClosureUnavailableOutcome::StageWithoutClosure); +} diff --git a/tests/unit/test_pack_route.cpp b/tests/unit/test_pack_route.cpp new file mode 100644 index 000000000..92dcf8485 --- /dev/null +++ b/tests/unit/test_pack_route.cpp @@ -0,0 +1,64 @@ +#include + +import std; +import mcpp.pack.route; + +using namespace mcpp::pack; + +// #630 A9: the route is chosen by the artifact's FORM, not by the target's +// kind. `accepts_several_targets` is the pure predicate `cmd_pack` asks +// before it builds anything — a program route serves more than one +// `--target` only when EVERY requested row resolves to a shared-object form, +// which today means every row is an Android row and the target is a +// `kind = "app"` (`PackRoute::isApplication`). + +namespace { + +PackRoute app_route() { return PackRoute{ "myapp", /*library=*/false, /*isApplication=*/true }; } +PackRoute bin_route() { return PackRoute{ "myapp", /*library=*/false, /*isApplication=*/false }; } +PackRoute library_route() { return PackRoute{ "mylib", /*library=*/true, /*isApplication=*/false }; } + +} // namespace + +TEST(AcceptsSeveralTargets, AnAppOnSeveralAndroidRowsIsAccepted) { + std::vector triples{"aarch64-linux-android", "x86_64-linux-android"}; + EXPECT_TRUE(accepts_several_targets(app_route(), triples)); +} + +TEST(AcceptsSeveralTargets, AnAppMixedWithAnExecutableRowIsRefused) { + // The negative direction inside "an app": one Android row and one row + // whose form is an executable must not slip through as "every row is a + // shared object" — a mixed request is not a request this route can + // satisfy either. + std::vector triples{"aarch64-linux-android", "x86_64-linux-gnu"}; + EXPECT_FALSE(accepts_several_targets(app_route(), triples)); +} + +TEST(AcceptsSeveralTargets, AnAppOnExecutableRowsOnlyIsRefused) { + std::vector triples{"x86_64-linux-gnu", "aarch64-macos"}; + EXPECT_FALSE(accepts_several_targets(app_route(), triples)); +} + +TEST(AcceptsSeveralTargets, ABinTargetIsAlwaysRefusedEvenOnAndroidRows) { + // `kind = "bin"` never resolves to a shared-object form on any row + // (`toolchain::triple::application_form` only answers `SharedObject` for + // `kind = "app"`), so the several-triple route is not this target's, + // whatever triples are named. + std::vector triples{"aarch64-linux-android", "x86_64-linux-android"}; + EXPECT_FALSE(accepts_several_targets(bin_route(), triples)); +} + +TEST(AcceptsSeveralTargets, AnUnparseableTripleIsRefusedNotIgnored) { + std::vector triples{"aarch64-linux-android", "not a triple"}; + EXPECT_FALSE(accepts_several_targets(app_route(), triples)); +} + +TEST(AcceptsSeveralTargets, ALibraryRouteIsUnaffected) { + // `accepts_several_targets` is never consulted for a library route in + // `cmd_pack` (libraries already take several triples unconditionally); + // asked anyway, it answers false because `isApplication` is false for a + // library `PackRoute`, which keeps the predicate honest about what it + // actually decides. + std::vector triples{"aarch64-linux-android", "x86_64-linux-android"}; + EXPECT_FALSE(accepts_several_targets(library_route(), triples)); +} diff --git a/tests/unit/test_pack_stage_tree.cpp b/tests/unit/test_pack_stage_tree.cpp index 08bfc73bc..90c0e428f 100644 --- a/tests/unit/test_pack_stage_tree.cpp +++ b/tests/unit/test_pack_stage_tree.cpp @@ -130,6 +130,62 @@ TEST(PackStageTree, TheEngineOwnsExactlyTwoFormatNames) { EXPECT_EQ(mcpp::pack::kBuiltinPackFormats.size(), 2u); } +// ── #630 §3: the `closure` field ───────────────────────────────────────── +// +// The "not-walked" case (a Mach-O program, or a non-PE artifact on a Windows +// host) cannot be produced on this (Linux) runner through `pack::run` -- it +// is covered end to end on macOS/iOS CI instead (see the e2e fixture's +// comment). What CAN be tested here, on every host, is the writer/reader +// contract itself: both values round-trip through the manifest file. + +TEST(PackStageTree, DefaultClosureIsWalkedAndSaysSo) { + Tmp t; + auto stage = t.path / "app"; + write_file(stage / "bin" / "app", "x"); + ASSERT_TRUE(mcpp::pack::write_stage_manifest(stage)); + auto text = read_file(mcpp::pack::stage_manifest_path(stage)); + EXPECT_EQ(text.substr(0, text.find('\n')), "closure = walked"); + EXPECT_EQ(text.find("reason ="), std::string::npos); +} + +TEST(PackStageTree, NotWalkedRecordsTheReasonOnOneLine) { + Tmp t; + auto stage = t.path / "app"; + write_file(stage / "bin" / "app", "x"); + mcpp::pack::ClosureStatus closure{ + .walked = false, + .reason = "cannot package the Mach-O program 'app' yet.\n" + " The dependency closure for that format is resolved " + "by running the artifact under\n" + " the target's own dynamic linker.", + }; + ASSERT_TRUE(mcpp::pack::write_stage_manifest(stage, closure)); + auto text = read_file(mcpp::pack::stage_manifest_path(stage)); + auto firstLine = text.substr(0, text.find('\n')); + EXPECT_EQ(firstLine, "closure = not-walked"); + // Folded to ONE line -- the manifest is one entry per line, and the + // reason text carries its own embedded newlines because it doubles as a + // CLI diagnostic. + auto secondLineEnd = text.find('\n', text.find('\n') + 1); + auto secondLine = text.substr(text.find('\n') + 1, secondLineEnd - text.find('\n') - 1); + EXPECT_TRUE(secondLine.starts_with("reason = cannot package the Mach-O program")); + EXPECT_EQ(secondLine.find('\n'), std::string::npos); + // The rest of the manifest -- the staged file listing -- is unaffected. + EXPECT_NE(text.find("bin/app"), std::string::npos); +} + +TEST(PackStageTree, WalkedAndNotWalkedProduceDifferentManifests) { + Tmp t; + auto stage = t.path / "app"; + write_file(stage / "bin" / "app", "x"); + ASSERT_TRUE(mcpp::pack::write_stage_manifest(stage, mcpp::pack::ClosureStatus{true, ""})); + auto walked = read_file(mcpp::pack::stage_manifest_path(stage)); + ASSERT_TRUE(mcpp::pack::write_stage_manifest( + stage, mcpp::pack::ClosureStatus{false, "no such linker to ask"})); + auto notWalked = read_file(mcpp::pack::stage_manifest_path(stage)); + EXPECT_NE(walked, notWalked); +} + TEST(PackStageTree, AMissingTreeIsRefusedRatherThanDescribedAsEmpty) { Tmp t; // An empty manifest for a directory that does not exist would say "nothing diff --git a/tests/unit/test_target_scalar_keys.cpp b/tests/unit/test_target_scalar_keys.cpp new file mode 100644 index 000000000..519f7f01c --- /dev/null +++ b/tests/unit/test_target_scalar_keys.cpp @@ -0,0 +1,179 @@ +#include + +import std; + +// #610 / #630 item 5 — `[target.]` has a scalar/array parser (a +// sequence of `body.find("")` blocks) and, further down in the same +// function, an unknown-key sweep that reports anything not named in +// `kKnownTargetScalars` / `kKnownTargetArrays`. The two lists are hand-written +// text living next to hand-written parse calls, and they drifted once +// already: `min_api_level` was added to the parser without being added to the +// sweep's lists, so a manifest that spelled it correctly was told the key was +// unsupported, and `--strict` turned that into a build failure. +// +// A test that hand-writes its own "the known keys are: ..." list would only +// re-encode the same assumption the sweep's list encodes, and would drift +// with it. Instead this test reads `modules/manifest/src/toml.cppm` as text +// and derives the set of parsed keys from the same `body.find("")` calls +// the parser executes, then checks it against the sweep's lists in both +// directions: +// +// - every parsed non-table key is named in the matching known-list, so a +// sixth key added to the parser without a matching list entry fails +// here instead of shipping silently; +// - every entry in a known-list has a matching parse site, so a stale or +// misspelled list entry (one that names nothing the parser reads) is +// caught too. +// +// The denominator — the set of `body.find(...)` calls — comes from the +// source tree, not from a list re-typed in this file, which is what makes +// the test unable to go stale the same way the code did. + +namespace { + +std::string read_file(const std::filesystem::path& p) { + std::ifstream input(p, std::ios::binary); + return std::string((std::istreambuf_iterator(input)), {}); +} + +std::filesystem::path repo_root() { + // tests/unit/test_target_scalar_keys.cpp -> tests/unit -> tests -> repo root + return std::filesystem::path(__FILE__).parent_path().parent_path().parent_path(); +} + +// The region of toml.cppm that both the scalar/array parser and the +// unknown-key sweep operate on: from the start of the `[target.]` +// loop body up to the assignment that closes it, `m.targetOverrides[...] = +// std::move(e);`. Bounding the scan to this region keeps it from picking up +// unrelated `body.find(...)` calls in the conditional-config channel that +// follows in the same function (`abi`, `runtime`, `build`, `xlings`, ...), +// which read the same variable name for a different table. +std::string target_entry_region(const std::string& source) { + auto begin = source.find("for (auto& [triple, val] : *tt) {"); + auto end = source.find("m.targetOverrides[canon_triple(triple)] = std::move(e);"); + if (begin == std::string::npos || end == std::string::npos || end < begin) return {}; + return source.substr(begin, end - begin); +} + +enum class Kind { Scalar, Array, Table }; + +// Classifies a `body.find("key")` call by which type-check appears first +// after it: the parser always tests the found value's type before reading +// it, and does so with `is_table()`, `is_array()`, `is_string()` or +// `is_int()` — `min_api_level` uses the last of these, everything else +// currently uses one of the first three. +Kind classify(const std::string& region, std::size_t keyPos, std::size_t nextKeyPos) { + std::size_t window_end = std::min(nextKeyPos, region.size()); + auto first_of = [&](std::string_view needle) { + auto p = region.find(needle, keyPos); + return (p == std::string::npos || p >= window_end) ? std::string::npos : p; + }; + std::size_t table = first_of("is_table()"); + std::size_t array = first_of("is_array()"); + std::size_t string_ = first_of("is_string()"); + std::size_t int_ = first_of("is_int()"); + std::size_t scalar = std::min(string_, int_); + if (table != std::string::npos && table < array && table < scalar) return Kind::Table; + if (array != std::string::npos && array < scalar) return Kind::Array; + return Kind::Scalar; +} + +struct ParsedKeys { + std::vector scalars; + std::vector arrays; +}; + +ParsedKeys parsed_target_keys(const std::string& region) { + ParsedKeys out; + const std::string needle = "body.find(\""; + std::size_t pos = 0; + std::vector> hits; + while ((pos = region.find(needle, pos)) != std::string::npos) { + std::size_t nameStart = pos + needle.size(); + std::size_t nameEnd = region.find('"', nameStart); + hits.emplace_back(pos, region.substr(nameStart, nameEnd - nameStart)); + pos = nameEnd; + } + for (std::size_t i = 0; i < hits.size(); ++i) { + std::size_t nextPos = (i + 1 < hits.size()) ? hits[i + 1].first : region.size(); + Kind k = classify(region, hits[i].first, nextPos); + const std::string& key = hits[i].second; + switch (k) { + case Kind::Scalar: out.scalars.push_back(key); break; + case Kind::Array: out.arrays.push_back(key); break; + case Kind::Table: break; // sub-tables are exempt from the sweep + } + } + return out; +} + +// Pulls the string literals out of `kKnownTargetScalars[] = { "a", "b", };` +// (or the analogous `kKnownTargetArrays`), the sweep's own known-key lists. +std::vector known_list(const std::string& region, std::string_view arrayName) { + std::vector out; + auto declPos = region.find(arrayName); + if (declPos == std::string::npos) return out; + auto braceStart = region.find('{', declPos); + auto braceEnd = region.find('}', braceStart); + std::string body = region.substr(braceStart + 1, braceEnd - braceStart - 1); + std::size_t pos = 0; + while ((pos = body.find('"', pos)) != std::string::npos) { + auto end = body.find('"', pos + 1); + out.push_back(body.substr(pos + 1, end - pos - 1)); + pos = end + 1; + } + return out; +} + +} // namespace + +TEST(TargetScalarKeys, EveryParsedNonTableKeyIsKnownToTheSweep) { + auto source = read_file(repo_root() / "modules" / "manifest" / "src" / "toml.cppm"); + ASSERT_FALSE(source.empty()) << "could not read toml.cppm"; + auto region = target_entry_region(source); + ASSERT_FALSE(region.empty()) << "could not locate the [target.] entry parser"; + + auto parsed = parsed_target_keys(region); + auto knownScalars = known_list(region, "kKnownTargetScalars"); + auto knownArrays = known_list(region, "kKnownTargetArrays"); + ASSERT_FALSE(knownScalars.empty()); + ASSERT_FALSE(knownArrays.empty()); + + // Direction 1 (the positive claim, and the failure mode #610 produced): + // every key the parser reads as a scalar is in the sweep's scalar list, + // and likewise for arrays. `min_api_level` is the case that used to fail + // this. + for (auto const& key : parsed.scalars) + EXPECT_NE(std::ranges::find(knownScalars, key), knownScalars.end()) + << "'" << key << "' is parsed as a scalar but missing from " + << "kKnownTargetScalars, so the sweep would report it as unsupported"; + for (auto const& key : parsed.arrays) + EXPECT_NE(std::ranges::find(knownArrays, key), knownArrays.end()) + << "'" << key << "' is parsed as an array but missing from " + << "kKnownTargetArrays"; + + // Direction 2 (the negative claim): every entry in a known-list actually + // names something the parser reads. A stale list entry would pass every + // manifest silently and give no test failure otherwise. + for (auto const& key : knownScalars) + EXPECT_NE(std::ranges::find(parsed.scalars, key), parsed.scalars.end()) + << "kKnownTargetScalars names '" << key << "', which has no " + << "body.find(...) parse site in the [target.] entry parser"; + for (auto const& key : knownArrays) + EXPECT_NE(std::ranges::find(parsed.arrays, key), parsed.arrays.end()) + << "kKnownTargetArrays names '" << key << "', which has no " + << "body.find(...) parse site"; +} + +TEST(TargetScalarKeys, MinApiLevelIsParsedAndKnown) { + // A direct, non-derived check on the specific regression: `min_api_level` + // must be both a parse site and a known scalar key. If this test passes + // while the source-scan test above also passes, the fix is coherent; if + // this one passes and the scan test fails, a *different* key drifted. + auto source = read_file(repo_root() / "modules" / "manifest" / "src" / "toml.cppm"); + auto region = target_entry_region(source); + ASSERT_FALSE(region.empty()); + EXPECT_NE(region.find("body.find(\"min_api_level\")"), std::string::npos); + auto knownScalars = known_list(region, "kKnownTargetScalars"); + EXPECT_NE(std::ranges::find(knownScalars, std::string("min_api_level")), knownScalars.end()); +} diff --git a/tests/unit/test_targetside.cpp b/tests/unit/test_targetside.cpp index 6893452fb..30c7f4fd5 100644 --- a/tests/unit/test_targetside.cpp +++ b/tests/unit/test_targetside.cpp @@ -198,6 +198,50 @@ TEST(TargetSideResolve, PrebuiltCLibraryUnderAGraphSuppliedCxxSubset) { EXPECT_TRUE(r.cxx.subset) << "no std module declared, so the library is a subset"; } +// A hosted target whose C library is the payload's (a located SDK, or glibc) +// while a package supplies the whole standard library: the iOS rows over +// `llvm.libcxx` (mcpp#630). The C++ layer is the graph's and is not a subset, +// and the C library stays the payload's; nothing about one decides the other. +TEST(TargetSideResolve, PrebuiltCLibraryUnderAGraphSuppliedLibcxx) { + auto in = payload_linux(); + in.cxxAbi = provider("libcxx", "22.1.8.1", "libc++", /*stdModule=*/true); + + auto r = ts::resolve(in); + EXPECT_EQ(r.cAbi.origin, ts::Origin::Payload) << "the SDK's or glibc's, prebuilt"; + EXPECT_TRUE(r.cAbi.prebuilt()); + EXPECT_EQ(r.cxx.origin, ts::Origin::Graph); + EXPECT_TRUE(r.cxx.fromGraph()); + EXPECT_FALSE(r.cxx.subset) << "a std module is declared, so this is the whole library"; + EXPECT_FALSE(r.system_from_graph()) << "the system is still the payload's"; +} + +// The payload has no compiler runtime for this platform and the graph +// declares none: the layer is absent, and the report says so rather than +// naming the compiler's family for an archive that does not exist. With a +// graph provider the same input resolves to the graph. +TEST(TargetSideResolve, APayloadWithoutACompilerRuntimeForThePlatform) { + auto in = payload_linux(); + in.llvmTriple = "arm64-apple-ios18.0-simulator"; + in.targetOs = "ios"; + in.targetEnv = "sim"; + in.compilerFamily = "llvm"; + in.payloadCompilerRuntimeAbsent = true; + + auto absent = ts::resolve(in); + EXPECT_TRUE(absent.compilerRuntime.absent()); + + in.compilerRuntime = provider("compiler-rt-builtins", "22.1.8.3", "compiler-rt"); + auto supplied = ts::resolve(in); + EXPECT_EQ(supplied.compilerRuntime.origin, ts::Origin::Graph); + EXPECT_FALSE(supplied.compilerRuntime.absent()); + + // Negative direction: a payload that has the archive reports its own. + in.compilerRuntime.reset(); + in.payloadCompilerRuntimeAbsent = false; + auto own = ts::resolve(in); + EXPECT_EQ(own.compilerRuntime.origin, ts::Origin::Payload); +} + TEST(TargetSideResolve, ZeroLibcTierHasNothingAtAll) { ts::Inputs in; in.llvmTriple = "x86_64-none-elf"; diff --git a/tests/unit/test_tool_store.cpp b/tests/unit/test_tool_store.cpp new file mode 100644 index 000000000..9ed0e00c0 --- /dev/null +++ b/tests/unit/test_tool_store.cpp @@ -0,0 +1,75 @@ +// The tool store's stamp for a source tree that has no version of its own. +// +// A `path` tool package changes under an unchanged version, and the store +// keyed on the version alone served a binary built from sources that no longer +// existed (measured 2026-09-08, examples/12). The stamp is what ninja itself +// trusts: every regular file's relative path, size and modification time. The +// tests state the three properties the key needs: the same tree gives the same +// stamp, a change to a source moves it, and a change under the build products +// does not. + +#include +#include + +import std; +import mcpp.build.tool_store; + +namespace fs = std::filesystem; + +namespace { + +struct Tree { + fs::path root; + explicit Tree(std::string_view name) : root(fs::temp_directory_path() / name) { + fs::remove_all(root); + fs::create_directories(root / "src"); + fs::create_directories(root / "target" / "x"); + write("mcpp.toml", "[package]\nname = \"t\"\nversion = \"0.1.0\"\n"); + write("src/gen.cpp", "int main() { return 41; }\n"); + write("target/x/gen.o", "object"); + } + ~Tree() { std::error_code ec; fs::remove_all(root, ec); } + void write(std::string_view rel, std::string_view content) { + std::ofstream(root / rel) << content; + } +}; + +// Two writes in one second have equal mtimes at second resolution on some +// filesystems; a size change is the part of the stamp that cannot be +// masked by that, so every edit below also changes the length. +} // namespace + +TEST(ToolStoreStamp, TheSameTreeGivesTheSameStamp) { + Tree t("mcpp_tool_store_stamp_same"); + const auto a = mcpp::build::tool_store::tree_stamp(t.root); + const auto b = mcpp::build::tool_store::tree_stamp(t.root); + EXPECT_EQ(a, b); + EXPECT_EQ(a.size(), 16u) << "sixteen hex digits, the fingerprint module's shape"; +} + +TEST(ToolStoreStamp, AnEditToASourceMovesTheStampInBothDirections) { + Tree t("mcpp_tool_store_stamp_edit"); + const auto before = mcpp::build::tool_store::tree_stamp(t.root); + t.write("src/gen.cpp", "int main() { return 42; } // edited\n"); + const auto edited = mcpp::build::tool_store::tree_stamp(t.root); + EXPECT_NE(before, edited); + // The reversal is a new state too: the file's length is back, its + // modification time is not, and a key that only ever grew would miss + // exactly this step. + t.write("src/gen.cpp", "int main() { return 41; }\n"); + const auto reverted = mcpp::build::tool_store::tree_stamp(t.root); + EXPECT_NE(edited, reverted); +} + +TEST(ToolStoreStamp, BuildProductsAndTheVersionControlDirectoryDoNotCount) { + Tree t("mcpp_tool_store_stamp_target"); + const auto before = mcpp::build::tool_store::tree_stamp(t.root); + t.write("target/x/gen.o", "a different object of another length"); + fs::create_directories(t.root / ".git"); + t.write(".git/HEAD", "ref: refs/heads/main\n"); + fs::create_directories(t.root / ".mcpp"); + t.write(".mcpp/stamp", "x"); + t.write("compile_commands.json", "[]"); + const auto after = mcpp::build::tool_store::tree_stamp(t.root); + EXPECT_EQ(before, after) << "build products, .git, .mcpp and the compile database are excluded"; +} diff --git a/tests/unit/test_xpkg_emit.cpp b/tests/unit/test_xpkg_emit.cpp index 4be1af654..74eb1d5c6 100644 --- a/tests/unit/test_xpkg_emit.cpp +++ b/tests/unit/test_xpkg_emit.cpp @@ -5,6 +5,7 @@ import mcpp.manifest; import mcpp.modgraph.graph; import mcpp.platform.env; import mcpp.publish.xpkg_emit; +import mcpp.diag; using namespace mcpp::publish; @@ -158,3 +159,85 @@ TEST(XpkgEmit, LongBracketSequenceInValueIsHarmless) { auto out = emit_xpkg(m, g, placeholder_release("0.1.0")); EXPECT_NE(out.find("\"trick: ]==] more stuff\""), std::string::npos); } + +// ── #630 item 7: an OS-only selector is a platform ────────────────────────── +// +// Design record 2026-09-13-630 §8.3's two criteria, both asserted here: an +// OS-only `[target.]` tool declaration names its block and raises +// no `publish/target-axis-tools` warning; a selector that is not OS-only +// names no block and still raises the warning (the residual case, so the +// advisory is not dead code once the OS-only path exists). + +namespace { + +// The three platform blocks are rendered back to back, each opened by its +// own header line and closed by " },\n" before the next one starts — +// see `mcpp::pm::emit_xpkg`. Slicing out one block lets a test ask "does +// THIS platform name the address" without the substring also matching a +// different block that happens to share indentation. +std::string block(const std::string& out, std::string_view header) { + auto p = out.find(header); + if (p == std::string::npos) return {}; + auto end = out.find(" },\n", p); + return out.substr(p, (end == std::string::npos ? out.size() : end) - p); +} + +} // namespace + +TEST(XpkgEmit, OsOnlySelectorNamesItsPlatformBlockAndSuppressesTheWarning) { + constexpr const char* src = R"( +[package] +name = "gtkapp" +version = "0.1.0" + +[target.'cfg(linux)'.xlings.workspace] +"xim:gtk4" = "" +)"; + auto m = mcpp::manifest::parse_string(src); + ASSERT_TRUE(m.has_value()) << (m ? std::string{} : m.error().format()); + auto g = minimal_graph(); + + mcpp::diag::reset(); + auto out = emit_xpkg(*m, g, placeholder_release("0.1.0")); + + EXPECT_NE(block(out, "linux = {").find("xim:gtk4"), std::string::npos) + << out; + EXPECT_EQ(block(out, "macosx = {").find("xim:gtk4"), std::string::npos); + EXPECT_EQ(block(out, "windows = {").find("xim:gtk4"), std::string::npos); + + // No `publish/target-axis-tools` advisory for THIS section: an OS-only + // selector is emitted, so there is nothing left to warn about. + EXPECT_EQ(mcpp::diag::count(mcpp::diag::Severity::Warning), 0u); + EXPECT_TRUE(mcpp::diag::records().empty()); +} + +TEST(XpkgEmit, NonOsSelectorNamesNoBlockAndStillWarns) { + constexpr const char* src = R"( +[package] +name = "gtkapp" +version = "0.1.0" + +[target.'cfg(target_arch = "aarch64")'.xlings.workspace] +"xim:gtk4" = "" +)"; + auto m = mcpp::manifest::parse_string(src); + ASSERT_TRUE(m.has_value()) << (m ? std::string{} : m.error().format()); + auto g = minimal_graph(); + + mcpp::diag::reset(); + auto out = emit_xpkg(*m, g, placeholder_release("0.1.0")); + + EXPECT_EQ(block(out, "linux = {").find("xim:gtk4"), std::string::npos); + EXPECT_EQ(block(out, "macosx = {").find("xim:gtk4"), std::string::npos); + EXPECT_EQ(block(out, "windows = {").find("xim:gtk4"), std::string::npos); + + // The residual case: the advisory still fires, and its text now also + // states that an OS-only selector WOULD have been emitted. + ASSERT_EQ(mcpp::diag::count(mcpp::diag::Severity::Warning), 1u); + auto recs = mcpp::diag::records(); + ASSERT_EQ(recs.size(), 1u); + EXPECT_EQ(recs.front().domain, "publish/target-axis-tools"); + EXPECT_NE(recs.front().what.find("xim:gtk4"), std::string::npos); + EXPECT_NE(recs.front().what.find("OS-only selector"), std::string::npos) + << recs.front().what; +}