From 3956f636361e0e48825f75e9282867992ed57354 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Mon, 14 Sep 2026 15:22:34 +0800 Subject: [PATCH 01/13] feat(dist-apk): read the closure the engine staged, pack several ABIs into one APK, add --format aab, and report every refusal as a warning (mcpp#634) --- dist/apk.cppm | 864 ++++++++++++++++++++++++++------------------------ mcpp.toml | 8 +- 2 files changed, 458 insertions(+), 414 deletions(-) diff --git a/dist/apk.cppm b/dist/apk.cppm index 866cf49..fdf1990 100644 --- a/dist/apk.cppm +++ b/dist/apk.cppm @@ -1,5 +1,5 @@ // mcpp.dist.apk -- an application target becomes an installable, signed -// `.apk`, with or without a Java host. +// `.apk`, or a signed Android App Bundle (`.aab`), with or without a Java host. // // WHY THIS IS NEITHER A RULE NOR A TOOL. A rule states how a translation unit // is compiled by a compiler mcpp does not drive. A tool states something the @@ -16,50 +16,41 @@ // code from a Java/Kotlin activity. Both tiers share every step below except // the two that compile and dex Java sources, which level 0 never submits. // -// FIVE ACTIONS, level 0, always in this order and under these ids -- `apk: -// manifest` (resource compilation, submitted only when `options::resources` -// names a directory -- there is nothing else this step could do without one), -// `apk:link` (aapt2 turns the generated manifest, `-I android.jar` and the -// optional compiled resources into an unsigned, unaligned `base.apk`), -// `apk:libs` (the native library and the deployed assets join the archive -- -// aapt2 has no flag for this, so this step is `jar`, not aapt2), `apk:align` -// (`zipalign`), `apk:sign` (`apksigner`). Level 1 adds `apk:javac` and -// `apk:d8` between `apk:link` and `apk:libs`, and `apk:libs`'s own command -// grows one more `-C` pair for `classes.dex`. +// THE STEPS, under these ids and in this order. `apk:manifest` compiles +// `options::resources` with aapt2 and is submitted only when that option names +// a directory. `apk:link` links the generated manifest, `-I android.jar` and +// the compiled resources into an unsigned, unaligned `base.apk`. Level 1 adds +// `apk:javac` and `apk:d8`. `apk:libs` adds the native libraries, the deployed +// assets and the dex with `jar` (aapt2 has no flag for native libraries), +// `apk:align` runs `zipalign` and `apk:sign` runs `apksigner`. // -// WHAT `mcpp pack`'S OWN CLOSURE DOES NOT DO FOR THIS ROW, MEASURED. Android's -// `run_shared_program` (mcpp.pack) stages the linked `.so` under the staged -// tree's `lib/` and stops there -- no dependency closure (the object cannot be -// executed on this host to ask it, `mcpp.pack.pack`'s own comment on that -// function says so) and, unlike every other row's `run()`, no call to -// `stage_runtime_files`: `mcpp::deploy`'s destinations are computed into -// `opts.runtimeFiles` on every row (`mcpp.pack.pipeline`) but the Android -// branch never consumes that vector. So a project's deployed resources exist -// only in the ORDINARY build output -- `${mcpp.out_dir}/bin//...`, beside -// the linked `.so` there, exactly where `mcpp::deploy`'s own contract puts -// them ("relative to the executable's directory") -- and never in -// `${mcpp.pack_stage_dir()}`'s `lib/`. This member therefore reads the -// program's own native library from the STAGED tree (`lib/*.so`, which is -// where a future closure would add more) and everything deployed from the -// BUILD tree's `bin/` (every subdirectory there that is not the link output -// itself), rather than from one single source the way `dist-appimage` and -// `dist-apple` can. Measured 2026-09-12 against this exact engine revision -// with a throwaway fixture; recorded here because the design record's own -// table row ("`.apk` | `assets/myapp.resources/`") reads as though the staged -// tree carried them, and it does not. +// `--format aab` SHARES EVERYTHING BUT THE LAST THREE STEPS. `aab:link` asks +// aapt2 for the protocol-buffer form bundletool reads (`--proto-format`); +// `aab:module` lays the linked archive out as a bundle's base module +// (`manifest/`, `dex/`, `lib//`, `assets/`); `aab:bundle` runs `bundletool +// build-bundle`; and `aab:sign` signs the bundle with `jarsigner`, because an +// App Bundle carries a JAR signature and not an APK signature scheme. // -// THE C++ RUNTIME IS SHARED, MEASURED. `readelf -d` of a NativeActivity -// `.so` built by this exact NDK payload (30.0.16248370, the one `xim: -// android-ndk` resolves for `*-linux-android` today) names `NEEDED -// libc++_shared.so` -- confirmed by the same warning `mcpp pack` itself -// prints on this row ("this toolchain ships no libc++.a/libc++abi.a; using -// toolchain-coupled"). The file is never in `${mcpp.pack_stage_dir()}`'s -// `lib/` (Android's closure does not walk it, see above), so this member -// takes it from the ACTIVE toolchain's own sysroot -- `mcpp::toolchain_dir()` -// resolves to `/toolchains/llvm/prebuilt/` for this row already, -// with no separate `xim:android-ndk` declaration needed on this member's own -// table, because the NDK is the toolchain building the project, not a tool -// this member wraps -- at `sysroot/usr/lib//libc++_shared.so`. +// THE NATIVE CLOSURE COMES FROM THE STAGED TREE (mcpp 2026.9.14.2+). The engine +// reads the application object's closure from the files and stages it: the +// object, every library the graph built for it, and the NDK's +// `libc++_shared.so` when the object needs it, under `lib/` for one triple or +// under `lib//` for several, with one `needs` line per name in the stage +// manifest (mcpp's docs/50, "The stage manifest"). This member copies those +// files into the package and reads the manifest to refuse a tree it cannot +// trust: an incomplete closure, a `needs` line whose staged file is absent, +// and a manifest with no `needs` line, which only an engine older than +// 2026.9.14.2 writes. Before that release the member walked `DT_NEEDED` itself +// at command time and recorded the walk in a stamp outside the staged tree, so +// a second pack of an unchanged project found the stamp current and produced a +// package without the dependency's library (measured on 0.9.3). The deployed +// files are staged under the tree's `bin/` on this row as on every other, and +// become `assets/`. +// +// EVERY REFUSAL IS ALSO A `mcpp::warning`. A member that refuses submits no +// action and its build program exits 0, and the engine discards the output of +// a build program that succeeded; its own error, "no action claimed --format +// 'apk'", then named no reason (measured on 0.9.3 with two triples). // // SIGNING, THROUGH THE PACKAGE MODEL. The default keystore is // `xim:android-debug-keystore`'s one `debug.keystore`, whose alias @@ -68,9 +59,9 @@ // release -- publishing them discloses nothing (see that package's own // header). A project that names `options::keystore` as a package (never a // path) is signing with a key it keeps out of every public index; the -// password reaches `apksigner` as `env:`, a token `apksigner` itself -// resolves against its own environment at run time, so this member never -// reads the secret. +// password reaches `apksigner` as `env:` and `jarsigner` as +// `-storepass:env `, tokens each tool resolves against its own +// environment at run time, so this member never reads the secret. // // WHAT THIS MEMBER DOES NOT DO. It does not run `mcpp run --format apk` -- // that is `adb-run`, a session `xim:android-platform-tools` registers, and @@ -172,7 +163,8 @@ struct options { // the tool does, at run time, in its own process. std::string keystore_password_env; - // Where the produced file lands. Empty means `/.apk`. + // Where the produced file lands. Empty means `/.apk`, or + // `/.aab` for `--format aab`. std::string output; std::string out_dir = std::string(mcpp::out_dir()); }; @@ -275,55 +267,6 @@ inline std::string abi_for() { return {}; } -// The NDK triple directory under `sysroot/usr/lib/`, which spells arm64 -// differently from the ABI name above (`aarch64-linux-android`, not -// `arm64-v8a`) -- two vocabularies for the one row, read from two different -// places upstream, not a choice this member makes. -inline std::string ndk_lib_triple_for() { - const std::string a = mcpp::target_arch(); - if (a == "aarch64") return "aarch64-linux-android"; - if (a == "x86_64") return "x86_64-linux-android"; - return {}; -} - -// Does `so` NEED `libc++_shared.so`? Read from the dynamic section with -// `llvm-readelf -d`, from the SAME toolchain that linked it -- the one -// `mcpp::toolchain_dir()` names for this build, not a host `readelf` this -// project never declared. Measured against this exact NDK (30.0.16248370): -// an ordinary `import std;` link NEEDs it (`readelf -d` on the fixture's own -// `.so` lists `NEEDED libc++_shared.so`, and `mcpp pack`'s own warning on -// this row -- "this toolchain ships no libc++.a/libc++abi.a; using -// toolchain-coupled" -- says the same thing from the flags side), so this -// is asked per file rather than assumed true for every build: a project -// that links `-static-libstdc++` or carries no C++ translation unit at all -// needs nothing extra, and copying the runtime in unconditionally would -// carry a library nothing in the APK opens. -inline bool needs_libcxx_shared(const std::string& toolchainDir, const std::string& so) { - const std::string readelf = (fs::path(toolchainDir) / "bin" / "llvm-readelf").string(); - if (!is_file(readelf) || !is_file(so)) return false; - const std::string cmd = "\"" + readelf + "\" -d \"" + so + "\" 2>/dev/null"; - // `popen` is POSIX and Windows spells it `_popen` -- this module compiles - // on every host (`tests/all-rules-compile`), even though `plan_for` - // refuses before reaching this call on every row but Android. -#if defined(_WIN32) - FILE* p = ::_popen(cmd.c_str(), "r"); -#else - FILE* p = ::popen(cmd.c_str(), "r"); -#endif - if (!p) return false; - bool found = false; - char line[512]; - while (std::fgets(line, sizeof line, p)) { - if (std::strstr(line, "libc++_shared.so")) { found = true; break; } - } -#if defined(_WIN32) - ::_pclose(p); -#else - ::pclose(p); -#endif - return found; -} - // Index loop, not a range-for: GCC 16.1.0 refuses to inline // `__normal_iterator>::operator*() const` when a // module interface unit that imports `std` also range-for's (or otherwise @@ -489,7 +432,8 @@ inline std::string default_manifest_template(bool has_code) { } // Checks a manifest template and substitutes it, or refuses (returning -// `false` with `reason` set) naming exactly what is wrong. +// `false` with `reason` and `message` set) naming exactly what is wrong; the +// caller reports `message` (see `refuse`). // // THIS CHECK IS `dist-apk`'S OWN, DELIBERATELY NOT `dist-web`'S. A manifest // has a closed, six-token vocabulary this member itself defines; a web page @@ -505,24 +449,24 @@ inline bool render_manifest(const std::string& templateText, bool has_code, const std::string& activityName, const std::string& libName, const std::string& minSdk, const std::string& targetSdk, const std::string& versionName, const std::string& versionCode, - std::string& out, std::string& reason) { + std::string& out, std::string& reason, + std::string& message) { for (auto const& tok : tokens_in(templateText)) { if (std::ranges::find(manifest_tokens(), tok) == manifest_tokens().end()) { - std::cerr << "mcpp.dist.apk: the manifest template names an unknown " - "token '{{" << tok << "}}' -- expected one of " - "application_id, label, activity, lib_name, min_sdk, " - "target_sdk, version_name, version_code\n"; + message = "mcpp.dist.apk: the manifest template names an unknown " + "token '{{" + tok + "}}'; expected one of application_id, " + "label, activity, lib_name, min_sdk, target_sdk, " + "version_name, version_code."; reason = "unknown manifest template token '" + tok + "'"; return false; } } for (auto const& tok : required_manifest_tokens(has_code)) { if (templateText.find("{{" + tok + "}}") == std::string::npos) { - std::cerr << "mcpp.dist.apk: the manifest template does not use " - "'{{" << tok << "}}', and assets/mcpp-run.json -- " - "which adb-run starts the application from -- is " - "written from the same value: add {{" << tok - << "}} to the template.\n"; + message = "mcpp.dist.apk: the manifest template does not use '{{" + tok + + "}}', and assets/mcpp-run.json, which adb-run starts the " + "application from, is written from the same value: add {{" + + tok + "}} to the template."; reason = "manifest template missing required token '" + tok + "'"; return false; } @@ -573,78 +517,210 @@ inline void collect_tree(const fs::path& src, const fs::path& dst, } } +// ─── The staged tree ─────────────────────────────────────────────────────── + +// Records a refusal three ways, because each reaches a different reader. The +// short `reason` is for a caller that inspects the plan; stderr is for a +// build program that exits non-zero; `mcpp::warning` is for the ordinary case, +// a member that refuses, submits nothing and lets the program exit 0 -- the +// engine discards the output of a build program that succeeded, and then +// reports only "no action claimed --format 'apk'" (measured on 0.9.3 with two +// triples). The warning channel is one line per directive, so line breaks in +// the message are folded into spaces. +inline plan& refuse(plan& p, std::string reason, const std::string& message) { + std::cerr << message << '\n'; + std::string folded; + folded.reserve(message.size()); + bool space = false; + for (std::size_t i = 0; i < message.size(); ++i) { + const char c = message[i]; + if (c == '\n' || c == '\r') { space = true; continue; } + if (space) { + if (c == ' ') continue; + folded += ' '; + space = false; + } + folded += c; + } + mcpp::warning(folded.c_str()); + p.reason = std::move(reason); + return p; +} + +// What the engine wrote beside the staged tree: `.stage-manifest` +// (mcpp's docs/50, "The stage manifest"). Only the header and the `needs` +// lines are read; the file list that follows them is not. +struct stage_manifest { + bool found = false; + bool walked = true; + std::string reason; + struct need { std::string name; std::string where; }; + std::vector needs; // `where`: a staged path, `platform` or `unresolved` +}; + +inline stage_manifest read_stage_manifest(std::string stage) { + stage_manifest m; + while (stage.size() > 1 && (stage.back() == '/' || stage.back() == '\\')) + stage.pop_back(); + std::ifstream in(stage + ".stage-manifest", std::ios::binary); + if (!in) return m; + m.found = true; + std::string line; + while (std::getline(in, line)) { + if (!line.empty() && line.back() == '\r') line.pop_back(); + if (line == "closure = not-walked") { m.walked = false; continue; } + if (line.starts_with("reason = ")) { m.reason = line.substr(9); continue; } + if (!line.starts_with("needs\t")) continue; + const auto second = line.find('\t', 6); + if (second == std::string::npos) continue; + m.needs.push_back({ line.substr(6, second - 6), line.substr(second + 1) }); + } + return m; +} + +// Android's own ABI names, the directory names a several-triple tree stages +// under (`lib//`) and an APK stores its native libraries under. +inline bool is_android_abi(std::string_view name) { + return name == "arm64-v8a" || name == "x86_64" || name == "armeabi-v7a" || name == "x86"; +} + +// One ABI's native libraries, as the staged tree carries them. +struct native_leg { + std::string abi; + std::vector libraries; // absolute paths, sorted +}; + +inline std::vector shared_objects_in(const fs::path& dir) { + std::vector out; + std::error_code ec; + for (auto& e : fs::directory_iterator(dir, ec)) { + if (ec) break; + if (e.is_regular_file(ec) && e.path().extension() == ".so") + out.push_back(e.path().string()); + } + std::ranges::sort(out); + return out; +} + // ─── Plan ────────────────────────────────────────────────────────────────── inline plan plan_for(options opt = {}) { plan p; const std::string requested = mcpp::pack_format(); - if (requested != "apk") { + if (requested != "apk" && requested != "aab") { p.reason = requested.empty() ? "this build is not packaging" - : std::format("--format {} was requested, not apk", requested); + : std::format("--format {} was requested, not apk or aab", requested); return p; } + const bool bundle = requested == "aab"; + // Step ids name the format, so a build log says which package a step made. + const auto id = [&](const char* apkId, const char* aabId) { return bundle ? aabId : apkId; }; if (const std::string env = mcpp::target_env(); env != "android") { - std::cerr << std::format( - "mcpp.dist.apk: an APK is an Android format, and this build's " - "target environment is '{}'.\n" - " build for a *-linux-android target", - env.empty() ? "unknown" : env) << '\n'; - p.reason = "not an Android target"; - return p; + return refuse(p, "not an Android target", std::format( + "mcpp.dist.apk: an {} is an Android format, and this build's " + "target environment is '{}'. Build for a *-linux-android target.", + bundle ? "AAB" : "APK", env.empty() ? "unknown" : env)); } const std::string stage = mcpp::pack_stage_dir(); if (stage.empty()) { - std::cerr << "mcpp.dist.apk: mcpp reported no staged tree. This " - "member needs mcpp 2026.9.11.1 or newer.\n"; - p.reason = "no staged tree"; - return p; + return refuse(p, "no staged tree", + "mcpp.dist.apk: mcpp reported no staged tree. This member needs " + "mcpp 2026.9.14.2 or newer."); } const std::string target = target_for(opt); if (target.empty()) { - std::cerr << "mcpp.dist.apk: no target to package. Set " - "`options::target` to the app target's name.\n"; - p.reason = "no target"; - return p; + return refuse(p, "no target", + "mcpp.dist.apk: no target to package. Set `options::target` to the " + "app target's name."); } - const std::string abi = abi_for(); - if (abi.empty()) { - std::cerr << std::format( - "mcpp.dist.apk: '{}' is not an Android ABI this member knows -- " - "expected aarch64 or x86_64 (mcpp::target_arch())", - mcpp::target_arch()) << '\n'; - p.reason = "unsupported ABI"; - return p; + // ── the native closure, read from the staged tree ───────────────────── + // + // The engine states the closure it staged in the stage manifest. A + // manifest without a single `needs` line comes from an engine that staged + // the application object alone, and packing that tree would produce a + // package whose object cannot load, so it is refused rather than packed. + const auto manifest = read_stage_manifest(stage); + if (!manifest.found) { + return refuse(p, "no stage manifest", std::format( + "mcpp.dist.apk: the staged tree {} has no stage manifest beside it. " + "This member reads the native closure the engine staged, which needs " + "mcpp 2026.9.14.2 or newer.", stage)); + } + if (manifest.needs.empty()) { + return refuse(p, "stage manifest without needs lines", std::format( + "mcpp.dist.apk: the stage manifest of {} states no `needs` line, so " + "the engine that staged it did not read the application's native " + "closure: its libraries would be missing from the package. This " + "member needs mcpp 2026.9.14.2 or newer, which stages the closure " + "under lib/ and states it in the manifest.", stage)); + } + if (!manifest.walked) { + std::string names; + for (auto const& n : manifest.needs) + if (n.where == "unresolved") names += (names.empty() ? "" : ", ") + n.name; + return refuse(p, "incomplete native closure", std::format( + "mcpp.dist.apk: the application's native closure is incomplete ({}), " + "so a package built from this tree would not load: {}", + names.empty() ? std::string("no name given") : names, + manifest.reason.empty() ? std::string("the stage manifest gives no reason") + : manifest.reason)); + } + for (auto const& n : manifest.needs) { + if (n.where == "platform") continue; + if (!is_file((fs::path(stage) / n.where).string())) { + return refuse(p, "a staged library is missing", std::format( + "mcpp.dist.apk: the stage manifest names {} for {}, and the " + "staged tree {} does not carry that file.", n.where, n.name, stage)); + } } - // ── the app's own shared object, from the staged tree's lib/ ────────── - const std::string stageLib = (fs::path(stage) / "lib").string(); - if (!is_dir(stageLib)) { - std::cerr << std::format( - "mcpp.dist.apk: the staged tree at {} carries no lib/ -- expected " - "the app target's shared object there ({})", stage, stageLib) << '\n'; - p.reason = "no lib/ in the staged tree"; - return p; + // One flat `lib/` for one triple, `lib//` for several. + const fs::path stageLib = fs::path(stage) / "lib"; + if (!is_dir(stageLib.string())) { + return refuse(p, "no lib/ in the staged tree", std::format( + "mcpp.dist.apk: the staged tree {} carries no lib/, where the engine " + "stages the app target's shared object and its closure.", stage)); } - std::vector soFiles; - { std::error_code ec; - for (auto& e : fs::directory_iterator(stageLib, ec)) { - if (ec) break; - if (e.is_regular_file(ec) && e.path().extension() == ".so") - soFiles.push_back(e.path().string()); - } + std::vector legs; + { + std::vector abis; + std::error_code ec; + for (auto& e : fs::directory_iterator(stageLib, ec)) { + if (ec) break; + if (e.is_directory(ec) && is_android_abi(e.path().filename().string())) + abis.push_back(e.path().filename().string()); + } + std::ranges::sort(abis); + for (auto const& abi : abis) + legs.push_back({ abi, shared_objects_in(stageLib / abi) }); + if (legs.empty()) { + const std::string abi = abi_for(); + if (abi.empty()) { + return refuse(p, "unsupported ABI", std::format( + "mcpp.dist.apk: '{}' is not an Android ABI this member knows; " + "expected aarch64 or x86_64 (mcpp::target_arch()).", + mcpp::target_arch())); + } + legs.push_back({ abi, shared_objects_in(stageLib) }); + } } - if (soFiles.empty()) { - std::cerr << std::format( - "mcpp.dist.apk: {} carries no .so at all -- expected the app " - "target's own shared object", stageLib) << '\n'; - p.reason = "no shared object in the staged tree"; - return p; + const std::string objectName = "lib" + target + ".so"; + for (auto const& leg : legs) { + const bool hasObject = std::ranges::any_of(leg.libraries, [&](const std::string& f) { + return fs::path(f).filename().string() == objectName; + }); + if (!hasObject) { + return refuse(p, "no application object in the staged tree", std::format( + "mcpp.dist.apk: the staged tree carries no {} for the {} ABI. " + "`options::target` names '{}', and the engine stages that " + "target's shared object under lib/.", objectName, leg.abi, target)); + } } if (opt.java_sources.empty() && !opt.activity.empty()) { @@ -656,57 +732,44 @@ inline plan plan_for(options opt = {}) { "android.app.NativeActivity and ignores it"); } if (!opt.java_sources.empty() && opt.activity.empty()) { - std::cerr << "mcpp.dist.apk: options::java_sources is set, so this " - "is a level-1 (Java-hosted) package, and options::" - "activity is required: the manifest has no other way " - "to name the launchable activity.\n"; - p.reason = "java_sources without activity"; - return p; + return refuse(p, "java_sources without activity", + "mcpp.dist.apk: options::java_sources is set, so this is a level-1 " + "(Java-hosted) package, and options::activity is required: the " + "manifest has no other way to name the launchable activity."); } const bool hasCode = !opt.java_sources.empty(); // ── the payloads this member declared ────────────────────────────── const std::string buildTools = mcpp::xpkg_dir("xim", "android-build-tools"); if (buildTools.empty()) { - std::cerr << std::format( - "mcpp.dist.apk: xim:android-build-tools was not found.\n" - " declare it under [target.'cfg(env = \"android\")'." - "feature-xlings.dist-apk] in the consuming project, or install " - "it directly.") << '\n'; - p.reason = "android-build-tools not found"; - return p; + return refuse(p, "android-build-tools not found", + "mcpp.dist.apk: xim:android-build-tools was not found. Declare it " + "under [target.'cfg(env = \"android\")'.feature-xlings.dist-apk] in " + "the consuming project, or install it directly."); } const std::string platformDir = mcpp::xpkg_dir("xim", "android-platform"); if (platformDir.empty()) { - std::cerr << "mcpp.dist.apk: xim:android-platform was not found " - "(declare it under [target.'cfg(env = \"android\")'." - "feature-xlings.dist-apk]).\n"; - p.reason = "android-platform not found"; - return p; + return refuse(p, "android-platform not found", + "mcpp.dist.apk: xim:android-platform was not found (declare it under " + "[target.'cfg(env = \"android\")'.feature-xlings.dist-apk])."); } const std::string androidJar = (fs::path(platformDir) / "android.jar").string(); if (!is_file(androidJar)) { - std::cerr << std::format( - "mcpp.dist.apk: {} does not exist -- {} does not look like an " - "Android platform payload", androidJar, platformDir) << '\n'; - p.reason = "android.jar not found"; - return p; + return refuse(p, "android.jar not found", std::format( + "mcpp.dist.apk: {} does not exist; {} does not look like an Android " + "platform payload.", androidJar, platformDir)); } const std::string targetSdk = api_level_from_platform_dir(platformDir); if (targetSdk.empty()) { - std::cerr << std::format( + return refuse(p, "no API level", std::format( "mcpp.dist.apk: could not read an API level from the resolved " - "xim:android-platform directory '{}'", platformDir) << '\n'; - p.reason = "no API level"; - return p; + "xim:android-platform directory '{}'.", platformDir)); } const std::string minSdk = mcpp::min_platform_version(); if (minSdk.empty()) { - std::cerr << "mcpp.dist.apk: mcpp::min_platform_version() is empty " - "on an Android target; this member needs mcpp " - "2026.9.12.2 or newer.\n"; - p.reason = "no min platform version"; - return p; + return refuse(p, "no min platform version", + "mcpp.dist.apk: mcpp::min_platform_version() is empty on an Android " + "target; this member needs mcpp 2026.9.12.2 or newer."); } // `apksigner`/`d8`: the JAVA_HOME/PATH wrapper `xim:android-build-tools` @@ -723,72 +786,85 @@ inline plan plan_for(options opt = {}) { std::pair{"aapt2", aapt2}, std::pair{"zipalign", zipalign}, std::pair{"apksigner", apksigner}, std::pair{"d8", d8}}) { if (!is_file(path)) { - std::cerr << std::format( - "mcpp.dist.apk: {} was not found at {} -- {} does not look " - "like the xim:android-build-tools payload", - name, path, buildTools) << '\n'; - p.reason = std::format("{} not found", name); - return p; + return refuse(p, std::format("{} not found", name), std::format( + "mcpp.dist.apk: {} was not found at {}; {} does not look like the " + "xim:android-build-tools payload.", name, path, buildTools)); } } - // `javac`/`jar`: NEITHER is part of `xim:android-build-tools` (only - // `apksigner` and `d8` are wrapped there, see that package's header) -- - // both come from `xim:jdk-temurin` directly, declared on this member's - // own table rather than assumed reachable through android-build-tools' - // runtime dependency, which provisions the JDK for ITS OWN wrappers and - // does not make it visible to a consumer's build program (docs/31, - // "declare the tool where it will be looked up"). + // `javac`/`jar`/`jarsigner`: NONE is part of `xim:android-build-tools` + // (only `apksigner` and `d8` are wrapped there, see that package's header) + // -- all three come from `xim:jdk-temurin` directly, declared on this + // member's own table rather than assumed reachable through + // android-build-tools' runtime dependency, which provisions the JDK for + // ITS OWN wrappers and does not make it visible to a consumer's build + // program (docs/31, "declare the tool where it will be looked up"). const std::string jdkHome = mcpp::xpkg_dir("xim", "jdk-temurin"); if (jdkHome.empty()) { - std::cerr << "mcpp.dist.apk: xim:jdk-temurin was not found " - "(declare it under [target.'cfg(env = \"android\")'." - "feature-xlings.dist-apk]).\n"; - p.reason = "jdk-temurin not found"; - return p; + return refuse(p, "jdk-temurin not found", + "mcpp.dist.apk: xim:jdk-temurin was not found (declare it under " + "[target.'cfg(env = \"android\")'.feature-xlings.dist-apk])."); } - const std::string javac = (fs::path(jdkHome) / "bin" / "javac").string(); - const std::string jar = (fs::path(jdkHome) / "bin" / "jar").string(); - for (auto const& [name, path] : {std::pair{"javac", javac}, std::pair{"jar", jar}}) { + const std::string javac = (fs::path(jdkHome) / "bin" / "javac").string(); + const std::string jar = (fs::path(jdkHome) / "bin" / "jar").string(); + const std::string jarsigner = (fs::path(jdkHome) / "bin" / "jarsigner").string(); + for (auto const& [name, path] : {std::pair{"javac", javac}, std::pair{"jar", jar}, + std::pair{"jarsigner", jarsigner}}) { if (!is_file(path)) { - std::cerr << std::format( - "mcpp.dist.apk: {} was not found at {} -- {} does not look " - "like a JDK payload", name, path, jdkHome) << '\n'; - p.reason = std::format("{} not found", name); - return p; + return refuse(p, std::format("{} not found", name), std::format( + "mcpp.dist.apk: {} was not found at {}; {} does not look like a " + "JDK payload.", name, path, jdkHome)); + } + } + + // `bundletool`: only an App Bundle needs it. The launcher `xim:bundletool` + // writes runs the JDK it was installed against. + std::string bundletool; + if (bundle) { + const std::string btDir = mcpp::xpkg_dir("xim", "bundletool"); + bundletool = btDir.empty() ? std::string() + : (fs::path(btDir) / "bin" / "bundletool").string(); + if (!is_file(bundletool)) { + return refuse(p, "bundletool not found", std::format( + "mcpp.dist.apk: an Android App Bundle is built by bundletool, and " + "xim:bundletool was not found{}. It is declared under " + "[target.'cfg(env = \"android\")'.feature-xlings.dist-apk].", + btDir.empty() ? std::string() : std::format(" at {}", bundletool))); } } // ── signing: a keystore, an alias and a password ─────────────────── + // + // `apksigner` takes a password as `pass:` or `env:`; + // `jarsigner` takes the same two as `-storepass ` or + // `-storepass:env `. Both spellings are derived from one decision. std::string keystoreFile, keystoreAlias, keystorePassArg; + std::vector jarsignerPass; if (opt.keystore.empty()) { const std::string ksDir = mcpp::xpkg_dir("xim", "android-debug-keystore"); if (ksDir.empty()) { - std::cerr << "mcpp.dist.apk: xim:android-debug-keystore was not " - "found (declare it under [target.'cfg(env = " - "\"android\")'.feature-xlings.dist-apk], or set " - "options::keystore).\n"; - p.reason = "android-debug-keystore not found"; - return p; + return refuse(p, "android-debug-keystore not found", + "mcpp.dist.apk: xim:android-debug-keystore was not found (declare " + "it under [target.'cfg(env = \"android\")'.feature-xlings.dist-apk], " + "or set options::keystore)."); } keystoreFile = (fs::path(ksDir) / "debug.keystore").string(); // The exact, published Android debug-signing convention -- not a // secret; see this member's header and `xim:android-debug-keystore`'s // own. - keystoreAlias = "androiddebugkey"; + keystoreAlias = "androiddebugkey"; keystorePassArg = "pass:android"; + jarsignerPass = { "-storepass", "android", "-keypass", "android" }; } else { auto colon = opt.keystore.find(':'); const std::string ns = colon == std::string::npos ? "" : opt.keystore.substr(0, colon); const std::string name = colon == std::string::npos ? opt.keystore : opt.keystore.substr(colon + 1); const std::string ksDir = mcpp::xpkg_dir(ns.c_str(), name.c_str()); if (ksDir.empty()) { - std::cerr << std::format( - "mcpp.dist.apk: keystore package '{}' was not found. Declare " - "it under [target.'cfg(env = \"android\")'.xlings.workspace] " - "in the consuming project.", opt.keystore) << '\n'; - p.reason = "keystore package not found"; - return p; + return refuse(p, "keystore package not found", std::format( + "mcpp.dist.apk: keystore package '{}' was not found. Declare it " + "under [target.'cfg(env = \"android\")'.xlings.workspace] in the " + "consuming project.", opt.keystore)); } std::vector candidates; { std::error_code ec; @@ -800,24 +876,22 @@ inline plan plan_for(options opt = {}) { } } if (candidates.size() != 1) { - std::cerr << std::format( - "mcpp.dist.apk: expected exactly one *.keystore/*.jks file in " - "{}, found {}", ksDir, candidates.size()) << '\n'; - p.reason = "ambiguous keystore package"; - return p; + return refuse(p, "ambiguous keystore package", std::format( + "mcpp.dist.apk: expected exactly one *.keystore/*.jks file in {}, " + "found {}.", ksDir, candidates.size())); } keystoreFile = candidates.front(); if (opt.keystore_alias.empty() || opt.keystore_password_env.empty()) { - std::cerr << "mcpp.dist.apk: options::keystore names a package, " - "so options::keystore_alias and options::" - "keystore_password_env are both required -- a " - "private key has no convention this member may " - "assume.\n"; - p.reason = "keystore alias/password not given"; - return p; + return refuse(p, "keystore alias/password not given", + "mcpp.dist.apk: options::keystore names a package, so " + "options::keystore_alias and options::keystore_password_env are " + "both required: a private key has no convention this member may " + "assume."); } keystoreAlias = opt.keystore_alias; keystorePassArg = "env:" + opt.keystore_password_env; + jarsignerPass = { "-storepass:env", opt.keystore_password_env, + "-keypass:env", opt.keystore_password_env }; } // ── the manifest and the run sidecar, written now (plan time) ───────── @@ -830,10 +904,8 @@ inline plan plan_for(options opt = {}) { const std::string tplPath = (fs::path(mcpp::manifest_dir()) / opt.manifest_template).string(); if (!is_file(tplPath)) { - std::cerr << std::format( - "mcpp.dist.apk: the manifest template {} was not found", tplPath) << '\n'; - p.reason = "manifest template not found"; - return p; + return refuse(p, "manifest template not found", std::format( + "mcpp.dist.apk: the manifest template {} was not found.", tplPath)); } // The template is declared so an edit to it reaches the graph -- the // one thing the ask's workaround (overwriting the manifest after @@ -849,83 +921,39 @@ inline plan plan_for(options opt = {}) { const std::string versionName = mcpp::package_version() ? mcpp::package_version() : ""; const std::string versionCode = version_code_for(versionName); - std::string manifestBytes; + std::string manifestBytes, manifestReason, manifestMessage; if (!render_manifest(manifestTemplateText, hasCode, appId, label, activityName, target, minSdk, targetSdk, versionName, versionCode, - manifestBytes, p.reason)) { - return p; + manifestBytes, manifestReason, manifestMessage)) { + return refuse(p, manifestReason, manifestMessage); } - const std::string manifestPath = (fs::path(opt.out_dir) / "dist-apk" / "AndroidManifest.xml").string(); + const fs::path outDir = fs::path(opt.out_dir) / (bundle ? "dist-aab" : "dist-apk"); + const std::string manifestPath = (outDir / "AndroidManifest.xml").string(); if (!write_if_different(manifestPath, manifestBytes)) { - std::cerr << std::format("mcpp.dist.apk: cannot write {}", manifestPath) << '\n'; - p.reason = "cannot write AndroidManifest.xml"; - return p; + return refuse(p, "cannot write AndroidManifest.xml", std::format( + "mcpp.dist.apk: cannot write {}.", manifestPath)); } // ── the temporary staging tree: lib//, assets/ ───────────────── // - // ASSETS, READ AT PLAN TIME AGAINST `pack_stage_dir()`, THE SAME WAY - // EVERY OTHER MEMBER READS ITS STAGED TREE (`dist-appimage`'s `is_file`, - // `dist-apple`'s identical reads). docs/30 ("Producing a distributable") - // and e2e 651/649 in the mcpp tree both show `mcpp::deploy`'s `to` - // landing at `/bin//...`, beside the packed executable - // -- the second pass of `mcpp pack --format apk` runs `plan_for` AFTER - // the tree is staged, so those files already exist on disk when this - // program reads them, exactly as `dist-appimage` reads `${mcpp.stage_ - // dir}`'s `AppRun` candidates. - // - // MEASURED ON THIS ROW, 2026-09-12, AGAINST THE ENGINE REVISION THIS - // MEMBER IS BUILT AGAINST: `*-linux-android`'s own staged tree carries - // `lib/.so` and NOTHING ELSE -- `run_shared_program` (mcpp.pack) - // stages the app's own object and stops, calling neither the ELF - // closure walk nor `stage_runtime_files` the way every other row's - // `run()` does (see this member's header, "WHAT `mcpp pack`'S OWN - // CLOSURE DOES NOT DO FOR THIS ROW"). So the loop below is written to - // the documented, cross-row contract and DOES fire wherever the engine - // actually stages `bin//...` beside the executable; on THIS row, - // today, `bin/` does not exist in the staged tree at all, and the loop - // is a no-op -- a deployed file is declared correctly and staged - // nowhere, which is the honest report of a gap in `run_shared_program` - // rather than in this member. See this member's own report for the - // measurement that found it. - const fs::path work = fs::path(opt.out_dir) / "dist-apk" / "stage"; + // Populated at plan time from the engine's staged tree, which the second + // pass of `mcpp pack` runs this program after. It is emptied first, so a + // library the graph no longer builds does not survive into the next + // package, and every file copied here is an input of the step that adds + // it to the archive. + const fs::path work = outDir / "stage"; { std::error_code ec; fs::remove_all(work, ec); } std::vector libInputs; - const fs::path libAbiDir = work / "lib" / abi; - for (auto const& so : soFiles) collect_tree(so, libAbiDir / fs::path(so).filename(), libInputs); - - // `libc++_shared.so`, WHEN THE CLOSURE NEEDS IT. See `needs_libcxx_shared` - // for the measurement. Not in `${mcpp.pack_stage_dir()}`'s `lib/` -- - // Android's own closure does not walk dependencies onto that tree at all - // (this member's header) -- so it comes from the ACTIVE toolchain's own - // sysroot, the same one that linked every `.so` this member just staged. - { - const std::string toolchainDir = mcpp::toolchain_dir(); - const std::string triple = ndk_lib_triple_for(); - if (!toolchainDir.empty() && !triple.empty()) { - const std::string libcxx = - (fs::path(toolchainDir) / "sysroot" / "usr" / "lib" / triple - / "libc++_shared.so").string(); - bool needed = false; - for (auto const& so : soFiles) - if (needs_libcxx_shared(toolchainDir, so)) { needed = true; break; } - if (needed) { - if (is_file(libcxx)) - collect_tree(libcxx, libAbiDir / "libc++_shared.so", libInputs); - else - mcpp::warning(std::format( - "mcpp.dist.apk: the closure NEEDs libc++_shared.so but " - "{} does not exist -- the apk will fail to load", - libcxx).c_str()); - } - } - } + for (auto const& leg : legs) + for (auto const& so : leg.libraries) + collect_tree(so, work / "lib" / leg.abi / fs::path(so).filename(), libInputs); const fs::path assetsDir = work / "assets"; std::vector assetInputs; - { // every deploy'd file, `/bin/` -> `assets/` -- see the - // long comment above for what this loop finds on this row today. + { // every deploy'd file, `/bin/` -> `assets/`: the engine + // stages `mcpp::deploy`'s destinations under `bin/` on this row as on + // every other. const fs::path stageBin = fs::path(stage) / "bin"; std::error_code ec; if (fs::is_directory(stageBin, ec)) { @@ -939,87 +967,46 @@ inline plan plan_for(options opt = {}) { } const std::string runJsonPath = (assetsDir / "mcpp-run.json").string(); if (!write_if_different(runJsonPath, run_json(appId, activityName))) { - std::cerr << std::format("mcpp.dist.apk: cannot write {}", runJsonPath) << '\n'; - p.reason = "cannot write mcpp-run.json"; - return p; + return refuse(p, "cannot write mcpp-run.json", std::format( + "mcpp.dist.apk: cannot write {}.", runJsonPath)); } assetInputs.push_back(runJsonPath); - // ── the small helper script this pipeline needs (argv only, no - // shell): copies the archive and hands the result to `jar`. The - // staging tree it copies from (`work`) is fully populated by the time - // this runs -- native libraries and deployed assets alike are read - // above, at plan time, not discovered by this script -- so, unlike an - // earlier revision of this member, it takes no `bindir` argument at - // all ───────────────────────────────────────────────────────────── - const fs::path helpersDir = fs::path(opt.out_dir) / "dist-apk"; - const std::string copyThenJar = (helpersDir / "copy-then-jar.sh").string(); - write_if_different(copyThenJar, + // ── the small helper scripts this pipeline needs (argv only, no shell + // between the engine and the tool). Each is written at plan time, only + // when its bytes differ, and is marked executable. ───────────────────── + auto helper = [&](const char* name, std::string_view body) { + const std::string path = (outDir / name).string(); + write_if_different(path, body); + std::error_code ec; + fs::permissions(path, + fs::perms::owner_exec | fs::perms::group_exec | fs::perms::others_exec, + fs::perm_options::add, ec); + return path; + }; + // Copies the linked archive and adds the staged `lib/`, `assets/` and the + // dex with `jar`: aapt2 has no flag for native libraries. + const std::string copyThenJar = helper("copy-then-jar.sh", "#!/bin/sh\n" "# mcpp.dist.apk helper. Do not edit.\n" "set -e\n" "src=\"$1\"; dst=\"$2\"; jar=\"$3\"; shift 3\n" "cp \"$src\" \"$dst\"\n" "\"$jar\" uf \"$dst\" \"$@\"\n"); - { std::error_code ec; fs::permissions(copyThenJar, - fs::perms::owner_exec | fs::perms::group_exec | fs::perms::others_exec, - fs::perm_options::add, ec); } - // THE SHARED LIBRARIES THE GRAPH BUILT, WHICH THE STAGED TREE DOES NOT - // CARRY. A dependency declared `linkage = "shared"` is linked as its own - // `lib.so` beside the app's in the ordinary build's `bin/`, and the - // app's dynamic section NEEDs it by name. The engine stages the app's - // own object and the deployed files, but on this row the closure is - // `not-walked` (a host cannot run an Android artifact), so nothing else - // reaches `lib/` -- and an APK without `lib.so` installs and dies - // at `dlopen` (measured: HuxerUI's framework as a shared library, - // `NEEDED libhuxerui.so`, 2026.9.13.2 + 0.9.2). This helper walks NEEDED - // from the app's own object at command time -- the set is not knowable - // at plan time without the closure -- and copies every name that exists - // beside the object, recursively, into `lib//`; names that live - // nowhere beside it (the platform's `libandroid.so`, `libc.so`) are - // skipped, and `libc++_shared.so` is the plan-time copy above. - const std::string collectNeeded = (helpersDir / "collect-needed.sh").string(); - write_if_different(collectNeeded, - "#!/bin/sh\n" - "# mcpp.dist.apk helper. Do not edit.\n" - "set -e\n" - "readelf=\"$1\"; so=\"$2\"; dst=\"$3\"; stamp=\"$4\"\n" - "dir=$(dirname \"$so\")\n" - "mkdir -p \"$dst\"\n" - "copy_needed() {\n" - " \"$readelf\" -d \"$1\" | sed -n 's/.*(NEEDED).*\\[\\(.*\\)\\].*/\\1/p' | while IFS= read -r n; do\n" - " if [ -f \"$dir/$n\" ] && [ ! -f \"$dst/$n\" ]; then\n" - " cp \"$dir/$n\" \"$dst/$n\"\n" - " copy_needed \"$dir/$n\"\n" - " fi\n" - " done\n" - "}\n" - "copy_needed \"$so\"\n" - "mkdir -p \"$(dirname \"$stamp\")\"\n" - ": > \"$stamp\"\n"); - { std::error_code ec; fs::permissions(collectNeeded, - fs::perms::owner_exec | fs::perms::group_exec | fs::perms::others_exec, - fs::perm_options::add, ec); } - const std::string runAndStamp = (helpersDir / "run-and-stamp.sh").string(); - write_if_different(runAndStamp, + const std::string runAndStamp = helper("run-and-stamp.sh", "#!/bin/sh\n" "set -e\n" "stamp=\"$1\"; shift\n" "\"$@\"\n" "mkdir -p \"$(dirname \"$stamp\")\"\n" "touch \"$stamp\"\n"); - { std::error_code ec; fs::permissions(runAndStamp, - fs::perms::owner_exec | fs::perms::group_exec | fs::perms::others_exec, - fs::perm_options::add, ec); } // `d8` DOES NOT ACCEPT A DIRECTORY, MEASURED (2026-09-12, d8 9.2.4, // `xim:android-build-tools` 37.0.0): `d8 --output ` // fails in the tool itself, `Unsupported source file type`, one frame // into `BaseCommand$Builder.addProgramFiles`. javac's own output set is - // not knowable at plan time (see above), so this wrapper finds the - // `.class` files `d8`'s command line needs at COMMAND time instead -- - // the identical "find" this member already had to reach for `assets/`. - const std::string runD8 = (helpersDir / "run-d8.sh").string(); - write_if_different(runD8, + // not knowable at plan time, so this wrapper finds the `.class` files + // `d8`'s command line needs at COMMAND time instead. + const std::string runD8 = helper("run-d8.sh", "#!/bin/sh\n" "# mcpp.dist.apk helper. Do not edit.\n" "set -e\n" @@ -1030,24 +1017,38 @@ inline plan plan_for(options opt = {}) { " exit 1\n" "fi\n" "\"$d8\" \"$@\" $classes\n"); - { std::error_code ec; fs::permissions(runD8, - fs::perms::owner_exec | fs::perms::group_exec | fs::perms::others_exec, - fs::perm_options::add, ec); } + // THE BASE MODULE OF AN APP BUNDLE. `aapt2 link --proto-format` writes the + // manifest at the archive's root beside `resources.pb` and `res/`; + // bundletool reads a module whose manifest is under `manifest/`, whose dex + // is under `dex/` and whose native libraries and assets are under `lib/` + // and `assets/`. This lays the linked archive out that way. A level-0 + // package has no dex, which the fourth argument states as `-`: an empty + // argument does not survive the action's argv. + const std::string makeModule = bundle ? helper("make-module.sh", + "#!/bin/sh\n" + "# mcpp.dist.apk helper. Do not edit.\n" + "set -e\n" + "proto=\"$1\"; module=\"$2\"; work=\"$3\"; dex=\"$4\"; jar=\"$5\"; out=\"$6\"\n" + "rm -rf \"$module\" \"$out\"\n" + "mkdir -p \"$module/manifest\"\n" + "(cd \"$module\" && \"$jar\" xf \"$proto\")\n" + "mv \"$module/AndroidManifest.xml\" \"$module/manifest/AndroidManifest.xml\"\n" + "if [ -d \"$work/lib\" ]; then cp -R \"$work/lib\" \"$module/lib\"; fi\n" + "if [ -d \"$work/assets\" ]; then cp -R \"$work/assets\" \"$module/assets\"; fi\n" + "if [ \"$dex\" != - ]; then mkdir -p \"$module/dex\"; cp \"$dex\" \"$module/dex/classes.dex\"; fi\n" + "\"$jar\" cfM \"$out\" -C \"$module\" .\n") : std::string(); // ── the pipeline ──────────────────────────────────────────────────── - const fs::path outDir = fs::path(opt.out_dir) / "dist-apk"; std::vector assembled; // every step's own output, for later inputs if (!opt.resources.empty()) { if (!is_dir(opt.resources)) { - std::cerr << std::format( - "mcpp.dist.apk: options::resources '{}' is not a directory", - opt.resources) << '\n'; - p.reason = "resources directory not found"; - return p; + return refuse(p, "resources directory not found", std::format( + "mcpp.dist.apk: options::resources '{}' is not a directory.", + opt.resources)); } step compile; - compile.id = "apk:manifest"; + compile.id = id("apk:manifest", "aab:manifest"); compile.role = "artifact"; compile.description = "AAPT2 COMPILE"; compile.output = (outDir / "compiled.zip").string(); @@ -1058,12 +1059,25 @@ inline plan plan_for(options opt = {}) { } step link; - link.id = "apk:link"; + link.id = id("apk:link", "aab:link"); link.role = "artifact"; - link.description = "AAPT2 LINK"; - link.output = (outDir / "base.apk").string(); + link.description = bundle ? "AAPT2 LINK (PROTO)" : "AAPT2 LINK"; + link.output = (outDir / (bundle ? "base-proto.zip" : "base.apk")).string(); link.argv = { aapt2, "link", "-I", androidJar, "--manifest", manifestPath, "--min-sdk-version", minSdk, "--target-sdk-version", targetSdk }; + // The protocol-buffer form is the one bundletool reads; an APK is the + // binary form a device installs. A bundle also needs a version code, which + // bundletool refuses the base module without ("Version code not found in + // manifest", measured with 1.18.3) and the built-in manifest does not + // write: aapt2 injects the package version's into a manifest that states + // none, and leaves a template's own value alone. + if (bundle) { + link.argv.push_back("--proto-format"); + link.argv.push_back("--version-code"); link.argv.push_back(versionCode); + if (!versionName.empty()) { + link.argv.push_back("--version-name"); link.argv.push_back(versionName); + } + } // POSITIONAL, NOT `-R`. `-R` is aapt2's overlay: a compilation unit whose // resources must each override one the base already defines, and a // project's `res/` IS the base -- linked with `-R`, its first colour @@ -1075,7 +1089,6 @@ inline plan plan_for(options opt = {}) { link.inputs = { manifestPath, androidJar }; if (!assembled.empty()) link.inputs.push_back(assembled.back()); p.steps.push_back(link); - std::string apkPath = link.output; std::vector javaOutputs; // classes.dex, when level 1 if (hasCode) { @@ -1087,11 +1100,9 @@ inline plan plan_for(options opt = {}) { std::vector javaFiles; for (auto const& root : opt.java_sources) { if (!is_dir(root)) { - std::cerr << std::format( + return refuse(p, "java_sources directory not found", std::format( "mcpp.dist.apk: options::java_sources root '{}' is not a " - "directory", root) << '\n'; - p.reason = "java_sources directory not found"; - return p; + "directory.", root)); } const std::size_t before = javaFiles.size(); { std::error_code ec; @@ -1102,11 +1113,9 @@ inline plan plan_for(options opt = {}) { } } if (javaFiles.size() == before) { - std::cerr << std::format( - "mcpp.dist.apk: options::java_sources root '{}' carries " - "no .java file", root) << '\n'; - p.reason = "no .java sources"; - return p; + return refuse(p, "no .java sources", std::format( + "mcpp.dist.apk: options::java_sources root '{}' carries no " + ".java file.", root)); } // THE RE-RUN QUESTION (design record §3.3). `glob_fingerprint` // walks the PACKAGE ROOT and matches paths relative to it; a @@ -1131,7 +1140,7 @@ inline plan plan_for(options opt = {}) { const std::string classesDir = (outDir / "classes").string(); step javacStep; - javacStep.id = "apk:javac"; + javacStep.id = id("apk:javac", "aab:javac"); javacStep.role = "artifact"; javacStep.description = "JAVAC"; javacStep.output = classesDir + "/.stamp"; @@ -1145,7 +1154,7 @@ inline plan plan_for(options opt = {}) { const std::string dexDir = (outDir / "dex").string(); step d8Step; - d8Step.id = "apk:d8"; + d8Step.id = id("apk:d8", "aab:d8"); d8Step.role = "artifact"; d8Step.description = "D8"; d8Step.output = dexDir + "/classes.dex"; @@ -1156,28 +1165,58 @@ inline plan plan_for(options opt = {}) { javaOutputs.push_back(d8Step.output); } - // ── the graph's shared libraries, walked from the app's NEEDED ──────── - step needed; - needed.id = "apk:needed"; - needed.role = "artifact"; - needed.description = "APK NEEDED"; - needed.output = (outDir / "needed.stamp").string(); - { - const std::string toolchainDir = mcpp::toolchain_dir(); - const std::string readelf = (fs::path(toolchainDir) / "bin" / "llvm-readelf").string(); - const std::string targetFile = std::format("${{mcpp.target_file:{}}}", target); - needed.argv = { collectNeeded, readelf, targetFile, libAbiDir.string(), needed.output }; - needed.inputs = { targetFile }; + if (bundle) { + // ── an Android App Bundle: the base module, the bundle, the signature ─ + step baseModule; + baseModule.id = "aab:module"; + baseModule.role = "artifact"; + baseModule.description = "AAB BASE MODULE"; + baseModule.output = (outDir / "base.zip").string(); + baseModule.argv = { makeModule, link.output, (outDir / "module").string(), work.string(), + javaOutputs.empty() ? std::string("-") : javaOutputs.front(), + jar, baseModule.output }; + baseModule.inputs = { link.output }; + for (auto const& f : libInputs) baseModule.inputs.push_back(f); + for (auto const& f : assetInputs) baseModule.inputs.push_back(f); + for (auto const& f : javaOutputs) baseModule.inputs.push_back(f); + p.steps.push_back(baseModule); + + step build; + build.id = "aab:bundle"; + build.role = "artifact"; + build.description = "BUNDLETOOL BUILD-BUNDLE"; + build.output = (outDir / "unsigned.aab").string(); + build.argv = { bundletool, "build-bundle", "--modules=" + baseModule.output, + "--output=" + build.output, "--overwrite" }; + build.inputs = { baseModule.output }; + p.steps.push_back(build); + + step sign; + sign.id = "aab:sign"; + sign.role = "artifact"; + sign.description = "JARSIGNER"; + p.output = !opt.output.empty() ? opt.output + : (fs::path(opt.out_dir) / (target + ".aab")).string(); + sign.output = p.output; + sign.argv = { jarsigner, "-keystore", keystoreFile }; + for (auto const& a : jarsignerPass) sign.argv.push_back(a); + sign.argv.push_back("-signedjar"); sign.argv.push_back(sign.output); + sign.argv.push_back(build.output); + sign.argv.push_back(keystoreAlias); + sign.inputs = { build.output, keystoreFile }; + p.steps.push_back(sign); + + p.applies = true; + return p; } - p.steps.push_back(needed); - // ── native library and assets join the archive ───────────────────── + // ── native libraries, assets and dex join the archive ────────────── step libs; libs.id = "apk:libs"; libs.role = "artifact"; libs.description = "APK LIBS+ASSETS"; libs.output = (outDir / "withlibs.apk").string(); - libs.argv = { copyThenJar, apkPath, libs.output, jar, + libs.argv = { copyThenJar, link.output, libs.output, jar, "-C", work.string(), "lib", "-C", work.string(), "assets" }; if (!javaOutputs.empty()) { @@ -1185,8 +1224,8 @@ inline plan plan_for(options opt = {}) { libs.argv.push_back((outDir / "dex").string()); libs.argv.push_back("classes.dex"); } - libs.inputs = { apkPath, needed.output }; - for (auto const& f : libInputs) libs.inputs.push_back(f); + libs.inputs = { link.output }; + for (auto const& f : libInputs) libs.inputs.push_back(f); for (auto const& f : assetInputs) libs.inputs.push_back(f); for (auto const& f : javaOutputs) libs.inputs.push_back(f); p.steps.push_back(libs); @@ -1238,12 +1277,10 @@ inline bool submit(const plan& p) { a.submit(); } - // A FLOOR ON THIS MEMBER'S OWN OUTPUT, ON THE SUCCESS PATH. Nothing here - // reads the SIGNED apk (the tools have not run yet, only been declared -- - // see `dist/appimage.cppm`'s identical reasoning) so this checks what - // this program itself built into the staging tree: at least one native - // library besides bookkeeping. An APK that installs and starts nothing - // is the failure this whole category exists to catch. + // No floor on the output here: the signed package does not exist while + // this program runs (the tools have been declared, not run), and what an + // empty package would lack -- the application object -- is refused by + // `plan_for` before any step is planned. return true; } @@ -1251,6 +1288,7 @@ inline bool submit(const plan& p) { inline bool generate(options opt = {}) { mcpp::provides_pack_format("apk"); + mcpp::provides_pack_format("aab"); return submit(plan_for(std::move(opt))); } diff --git a/mcpp.toml b/mcpp.toml index 01c1289..462db7a 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -405,7 +405,7 @@ implies = ["surface"] # is downloaded" shape the accelerator rules use, applied to the target axis # instead. # -# FOUR PACKAGES, EACH OWNED BY A DIFFERENT HALF OF THE PIPELINE. `aapt2`, +# FOUR OF THE FIVE PACKAGES, EACH OWNED BY A DIFFERENT HALF OF THE PIPELINE. `aapt2`, # `zipalign`, and the `apksigner`/`d8` wrappers all live in one archive # (`xim:android-build-tools`); `android.jar` is versioned by API level on its # own (`xim:android-platform`) because that is the number a project pins, not @@ -430,11 +430,17 @@ implies = ["surface"] # directory this entry resolves to (`api_level_from_platform_dir`), so an # alias that silently failed to resolve would surface as a build-time # refusal rather than a wrong number, but there is no reason to court it. +# +# `xim:bundletool` BUILDS `--format aab`. It is installed with the other four +# for the reason `dist-appimage` gives above: provisioning runs before the +# build program learns `--format`, so the format cannot gate a download. Its +# launcher runs the JDK it was installed against, which is the one pinned here. [target.'cfg(env = "android")'.feature-xlings.dist-apk] "xim:android-build-tools" = ">=37.0.0" "xim:android-platform" = "36-r2" "xim:jdk-temurin" = "25.0.4+7" "xim:android-debug-keystore" = "1.0.0" +"xim:bundletool" = "1.18.3" [targets.plugins] kind = "lib" From 3c10cbcb180afe4b7778a249d50ddadb84fe8c1a Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Mon, 14 Sep 2026 16:02:50 +0800 Subject: [PATCH 02/13] feat(dist-apk): one stage-manifest reader in the lib root, and the closure criteria as a script (mcpp#634) `mcpp::plugins::stage::read_manifest` parses the header and the `needs` lines of `.stage-manifest`; dist-apk reads through it, and dist-apple is its second reader. tests/apk-consumer-shared/check-apk-closure.sh states five criteria: two packs in a row carry the dependency's library; two triples give one signed APK whose native code lists arm64-v8a and x86_64; --format aab gives a bundle that bundletool validates, jarsigner verifies, and that yields a universal APK; a refusal's reason is in mcpp pack's output; a stage manifest without `needs` lines is refused through mcpp::warning, naming 2026.9.14.2. Local readings (Linux, mcpp feat/634-cmake-parity at 99378fc): all five hold. Against the 0.9.3 member under the same engine, criteria 2, 4 and 5 fail (the two-triple pack exits 1 with "no action claimed"; the refusal's reason is absent; the build program plans nothing about the floor). --- dist/apk.cppm | 33 +--- src/plugins.cppm | 45 +++++ .../apk-consumer-shared/check-apk-closure.sh | 156 ++++++++++++++++++ tests/apk-consumer-shared/mcpp.toml | 11 +- 4 files changed, 208 insertions(+), 37 deletions(-) create mode 100755 tests/apk-consumer-shared/check-apk-closure.sh diff --git a/dist/apk.cppm b/dist/apk.cppm index fdf1990..be30da8 100644 --- a/dist/apk.cppm +++ b/dist/apk.cppm @@ -547,37 +547,6 @@ inline plan& refuse(plan& p, std::string reason, const std::string& message) { return p; } -// What the engine wrote beside the staged tree: `.stage-manifest` -// (mcpp's docs/50, "The stage manifest"). Only the header and the `needs` -// lines are read; the file list that follows them is not. -struct stage_manifest { - bool found = false; - bool walked = true; - std::string reason; - struct need { std::string name; std::string where; }; - std::vector needs; // `where`: a staged path, `platform` or `unresolved` -}; - -inline stage_manifest read_stage_manifest(std::string stage) { - stage_manifest m; - while (stage.size() > 1 && (stage.back() == '/' || stage.back() == '\\')) - stage.pop_back(); - std::ifstream in(stage + ".stage-manifest", std::ios::binary); - if (!in) return m; - m.found = true; - std::string line; - while (std::getline(in, line)) { - if (!line.empty() && line.back() == '\r') line.pop_back(); - if (line == "closure = not-walked") { m.walked = false; continue; } - if (line.starts_with("reason = ")) { m.reason = line.substr(9); continue; } - if (!line.starts_with("needs\t")) continue; - const auto second = line.find('\t', 6); - if (second == std::string::npos) continue; - m.needs.push_back({ line.substr(6, second - 6), line.substr(second + 1) }); - } - return m; -} - // Android's own ABI names, the directory names a several-triple tree stages // under (`lib//`) and an APK stores its native libraries under. inline bool is_android_abi(std::string_view name) { @@ -645,7 +614,7 @@ inline plan plan_for(options opt = {}) { // manifest without a single `needs` line comes from an engine that staged // the application object alone, and packing that tree would produce a // package whose object cannot load, so it is refused rather than packed. - const auto manifest = read_stage_manifest(stage); + const auto manifest = mcpp::plugins::stage::read_manifest(stage); if (!manifest.found) { return refuse(p, "no stage manifest", std::format( "mcpp.dist.apk: the staged tree {} has no stage manifest beside it. " diff --git a/src/plugins.cppm b/src/plugins.cppm index 604526c..d203fe2 100644 --- a/src/plugins.cppm +++ b/src/plugins.cppm @@ -53,6 +53,51 @@ inline constexpr std::string_view version = "0.9.3"; } // namespace mcpp::plugins +// mcpp::plugins::stage -- what the engine wrote beside a staged tree. +// +// SHARED BECAUSE TWO DIST MEMBERS READ IT. `mcpp pack` writes +// `.stage-manifest` (mcpp's docs/50, "The stage manifest"): a +// header, one `needs` line per library name the closure read (mcpp +// 2026.9.14.2+), and the list of staged files. `dist-apk` reads it to trust the +// native libraries under `lib/`, and `dist-apple` reads it to tell the dylibs +// staged beside a program from the resources staged there. Only the header and +// the `needs` lines are parsed; the file list is not. +export namespace mcpp::plugins::stage { + +struct need { + std::string name; // as the needing object spells it + std::string where; // a staged path relative to the tree, `platform` or `unresolved` +}; + +struct manifest { + bool found = false; // the file exists beside the tree + bool walked = true; // `closure = walked` + std::string reason; // `reason = ...`, with `closure = not-walked` + std::vector needs; +}; + +inline manifest read_manifest(std::string tree) { + manifest m; + while (tree.size() > 1 && (tree.back() == '/' || tree.back() == '\\')) + tree.pop_back(); + std::ifstream in(tree + ".stage-manifest", std::ios::binary); + if (!in) return m; + m.found = true; + std::string line; + while (std::getline(in, line)) { + if (!line.empty() && line.back() == '\r') line.pop_back(); + if (line == "closure = not-walked") { m.walked = false; continue; } + if (line.starts_with("reason = ")) { m.reason = line.substr(9); continue; } + if (!line.starts_with("needs\t")) continue; + const auto second = line.find('\t', 6); + if (second == std::string::npos) continue; + m.needs.push_back({ line.substr(6, second - 6), line.substr(second + 1) }); + } + return m; +} + +} // namespace mcpp::plugins::stage + // mcpp::plugins::names -- the derivations that turn a path into a C++ name. // // THESE ARE SHARED BECAUSE THEY WERE COPIED. `common_base_dir` and diff --git a/tests/apk-consumer-shared/check-apk-closure.sh b/tests/apk-consumer-shared/check-apk-closure.sh new file mode 100755 index 0000000..676390c --- /dev/null +++ b/tests/apk-consumer-shared/check-apk-closure.sh @@ -0,0 +1,156 @@ +#!/usr/bin/env bash +# dist-apk reads the native closure the engine staged (mcpp#634, B5). +# +# Five criteria. The first four run `mcpp pack` and read the package with the +# Android tools; the fifth runs the compiled build program against a fabricated +# stage, so it holds whichever engine packs. +# +# 1. Two packs in a row both carry the dependency's library. Under 0.9.3 the +# second APK lacked it: the member walked the object's NEEDED entries +# behind a stamp that outlived the stage the plan wipes, so the walk did +# not run again. +# 2. Two `--target` rows give one APK whose native code lists both ABIs, each +# with the application object, the dependency and `libc++_shared.so`. +# Under 0.9.3 this pack exited 1: the member did not descend into +# `lib//`. +# 3. `--format aab` gives an App Bundle that bundletool validates, that +# jarsigner verifies, and from which bundletool builds a universal APK +# carrying the same libraries. +# 4. A refusal reaches the user: `--format apk` for the host row prints the +# member's reason. Under 0.9.3 the reason went to the build program's +# stderr, which the engine discards when the program exits 0, and the +# user read only "no action claimed". +# 5. A stage manifest without `needs` lines, which every engine below +# 2026.9.14.2 writes, is refused through `mcpp::warning`, naming that +# floor. +# +# Usage: MCPP= ./check-apk-closure.sh (run from this directory) +set -eu + +MCPP="${MCPP:-mcpp}" +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +# The path of the artifact a pack reports on its `Packed` line. +packed() { sed -n 's/^ *Packed //p' "$1" | tail -1; } + +"$MCPP" self env > mcpp-env.txt +MCPP_HOME_DIR=$(awk -F'= *' '/^MCPP_HOME/{print $2; exit}' mcpp-env.txt) +[ -n "$MCPP_HOME_DIR" ] || fail "could not read MCPP_HOME" mcpp-env.txt +XPKGS="$MCPP_HOME_DIR/registry/data/xpkgs" + +# tool : the file in whichever installed +# version carries it. +tool() { + local d + for d in "$XPKGS/xim-x-$1"/*/; do + [ -e "$d$2" ] && { echo "$d$2"; return 0; } + done + return 1 +} + +# lists : the member names of a zip archive. +lists() { "$JAR" tf "$1" > "$2"; } + +# ── 1. two packs in a row ────────────────────────────────────────────────── +echo "== 1. two packs in a row carry the dependency's library ==" +rm -rf target +for n in 1 2; do + "$MCPP" pack --target x86_64-linux-android --format apk > "closure-pack$n.log" 2>&1 \ + || fail "pack $n exited non-zero" "closure-pack$n.log" + apk=$(packed "closure-pack$n.log") + [ -f "$apk" ] || fail "pack $n reported no APK" "closure-pack$n.log" + if [ "$n" = 1 ]; then + JAR=$(tool jdk-temurin bin/jar) || fail "no jar under $XPKGS/xim-x-jdk-temurin" + JARSIGNER=$(tool jdk-temurin bin/jarsigner) || fail "no jarsigner under $XPKGS/xim-x-jdk-temurin" + AAPT2=$(tool android-build-tools aapt2) || fail "no aapt2 under $XPKGS/xim-x-android-build-tools" + APKSIGNER=$(tool android-build-tools bin/apksigner) || fail "no apksigner under $XPKGS/xim-x-android-build-tools" + fi + lists "$apk" "closure-list$n.log" + for f in lib/x86_64/libapk-consumer-shared.so lib/x86_64/libapk-consumer-dep.so \ + lib/x86_64/libc++_shared.so; do + grep -qxF "$f" "closure-list$n.log" || fail "the APK of pack $n does not list $f" "closure-list$n.log" + done + echo "ok: pack $n: $(grep -c '^lib/' "closure-list$n.log") native libraries, the dependency's among them" +done + +# ── 2. two triples, one APK ──────────────────────────────────────────────── +echo "== 2. two triples give one APK listing both ABIs ==" +"$MCPP" pack --target x86_64-linux-android --target aarch64-linux-android --format apk \ + > closure-multi.log 2>&1 || fail "the two-triple pack exited non-zero" closure-multi.log +apk=$(packed closure-multi.log) +[ -f "$apk" ] || fail "the two-triple pack reported no APK" closure-multi.log +lists "$apk" closure-multi-list.log +for abi in x86_64 arm64-v8a; do + for f in libapk-consumer-shared.so libapk-consumer-dep.so libc++_shared.so; do + grep -qxF "lib/$abi/$f" closure-multi-list.log \ + || fail "the two-triple APK does not list lib/$abi/$f" closure-multi-list.log + done +done +"$AAPT2" dump badging "$apk" > closure-badging.log 2>&1 || fail "aapt2 could not read the APK" closure-badging.log +tr -d '\r' < closure-badging.log | grep -qx "native-code: 'arm64-v8a' 'x86_64'" \ + || fail "aapt2 does not report native-code 'arm64-v8a' 'x86_64'" closure-badging.log +"$APKSIGNER" verify "$apk" > closure-verify.log 2>&1 || fail "apksigner does not verify the APK" closure-verify.log +echo "ok: one signed APK, $(tr -d '\r' < closure-badging.log | grep '^native-code:')" + +# ── 3. an App Bundle ─────────────────────────────────────────────────────── +echo "== 3. --format aab ==" +"$MCPP" pack --target x86_64-linux-android --format aab > closure-aab.log 2>&1 \ + || fail "the aab pack exited non-zero" closure-aab.log +aab=$(packed closure-aab.log) +case "$aab" in *.aab) ;; *) fail "the aab pack reported '$aab', not an .aab" closure-aab.log ;; esac +[ -f "$aab" ] || fail "the reported bundle $aab does not exist" closure-aab.log +BUNDLETOOL=$(tool bundletool bin/bundletool) || fail "no bundletool under $XPKGS/xim-x-bundletool" +lists "$aab" closure-aab-list.log +for f in BundleConfig.pb base/manifest/AndroidManifest.xml base/resources.pb \ + base/lib/x86_64/libapk-consumer-shared.so base/lib/x86_64/libapk-consumer-dep.so \ + base/lib/x86_64/libc++_shared.so base/assets/mcpp-run.json; do + grep -qxF "$f" closure-aab-list.log || fail "the bundle does not list $f" closure-aab-list.log +done +"$BUNDLETOOL" validate --bundle="$aab" > closure-validate.log 2>&1 \ + || fail "bundletool validate refused the bundle" closure-validate.log +"$JARSIGNER" -verify "$aab" > closure-jarsigner.log 2>&1 || fail "jarsigner -verify failed" closure-jarsigner.log +grep -q '^jar verified' closure-jarsigner.log || fail "jarsigner did not report 'jar verified'" closure-jarsigner.log +work=$(mktemp -d) +"$BUNDLETOOL" build-apks --bundle="$aab" --output="$work/universal.apks" --mode=universal \ + > closure-build-apks.log 2>&1 || fail "bundletool build-apks --mode=universal failed" closure-build-apks.log +(cd "$work" && "$JAR" xf universal.apks universal.apk) || fail "the .apks set carries no universal.apk" +lists "$work/universal.apk" closure-universal-list.log +grep -qxF lib/x86_64/libapk-consumer-dep.so closure-universal-list.log \ + || fail "the universal APK does not carry the dependency's library" closure-universal-list.log +echo "ok: the bundle validates, is signed, and yields a universal APK with the dependency's library" + +# ── 4. a refusal reaches the user ────────────────────────────────────────── +echo "== 4. a refusal prints its reason ==" +rc=0 +"$MCPP" pack --format apk > closure-refusal.log 2>&1 || rc=$? +[ "$rc" -ne 0 ] || fail "an APK for the host row was not refused" closure-refusal.log +grep -q 'mcpp.dist.apk: an APK is an Android format' closure-refusal.log \ + || fail "the refusal's reason is not in mcpp pack's output (exit $rc)" closure-refusal.log +echo "ok: exit $rc, and the output names the reason: $(grep -m1 'mcpp.dist.apk:' closure-refusal.log)" + +# ── 5. an engine below the floor ─────────────────────────────────────────── +# +# The engine's output is not read here, so the build program is run with the +# environment a pack sets (mcpp's docs/30) and a stage manifest in the form an +# engine below 2026.9.14.2 writes: header and file list, no `needs` line. +echo "== 5. a stage manifest without needs lines is refused, naming the floor ==" +BIN=target/.build-mcpp/build.mcpp.bin +[ -x "$BIN" ] || fail "no compiled build.mcpp to run" closure-refusal.log +stage="$work/apk-consumer-shared-0.2.0-x86_64-linux-android" +mkdir -p "$stage/lib" +head -c 4096 /dev/zero > "$stage/lib/libapk-consumer-shared.so" +printf 'closure = not-walked\nreason = the Android closure is not read\n4096 lib/libapk-consumer-shared.so\n' \ + > "$stage.stage-manifest" +env -i PATH="$PATH" \ + MCPP_TARGET_ARCH=x86_64 MCPP_TARGET_OS=linux MCPP_TARGET_ENV=android \ + MCPP_OUT_DIR="$work/out" MCPP_MANIFEST_DIR="$PWD" \ + MCPP_PKG_NAME=apk-consumer-shared MCPP_PKG_VERSION=0.2.0 \ + MCPP_PACK_FORMAT=apk MCPP_PACK_STAGE_DIR="$stage" \ + "$BIN" > closure-floor.log 2>&1 || true +if grep -q '^mcpp:action=' closure-floor.log; then + fail "a stage without needs lines planned actions" closure-floor.log +fi +grep '^mcpp:warning=' closure-floor.log | grep -q 'no `needs` line.*2026\.9\.14\.2 or newer' \ + || fail "the refusal is not a warning naming mcpp 2026.9.14.2" closure-floor.log +echo "ok: refused, as a warning naming the engine floor" + +echo "PASS: dist-apk reads the staged closure, packs two ABIs into one APK, builds an App Bundle, and reports its refusals" diff --git a/tests/apk-consumer-shared/mcpp.toml b/tests/apk-consumer-shared/mcpp.toml index 4f818a1..2c82b52 100644 --- a/tests/apk-consumer-shared/mcpp.toml +++ b/tests/apk-consumer-shared/mcpp.toml @@ -1,10 +1,11 @@ # Fixture: an app target whose dependency is a SHARED object on the Android # row -- the shape a framework takes there (the Java host loads it by name), -# and the one `dist-apk` used to lose: the engine stages the app's own object -# and the deployed files, the closure on this row is `not-walked`, and -# `lib.so` reached neither `lib/` nor the APK (0.9.2). From 0.9.3 the -# member walks the app's NEEDED at command time and copies what it finds -# beside the object. +# and the one `dist-apk` used to lose. Under 0.9.2 `lib.so` reached +# neither `lib/` nor the APK; 0.9.3 walked the app's NEEDED at command time, +# behind a stamp that lost the library on a second pack. From mcpp 2026.9.14.2 +# the engine stages the closure under `lib/` (`lib//` for several targets) +# and names it in the stage manifest, and 0.10.0 packs that tree. +# `check-apk-closure.sh` states the criteria. [package] name = "apk-consumer-shared" version = "0.2.0" From 18099ca84744cde4706ea6c5e59dab901f432a62 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Mon, 14 Sep 2026 16:03:19 +0800 Subject: [PATCH 03/13] feat(dist-apple): the closure's dylibs in the framework directory, the link-time rpath, ad-hoc signing, the app runner and --format dmg (mcpp#634) The dylibs the engine stages beside a Mach-O program (2026.9.14.2+), named by the stage manifest's `needs` lines, are copied into Contents/Frameworks/ (Frameworks/ on iOS) and kept out of the resource directory. generate() adds -Wl,-rpath,@executable_path/../Frameworks on macOS and @executable_path/Frameworks on iOS through mcpp::link_flag, on every pass, because the link happens before the pass that learns --format; no load command is edited after the link. A macOS bundle without an identity is signed ad hoc, frameworks first and the bundle second; the iOS device row signs only with an identity and the simulator row never. An incomplete closure is a warning naming the unresolved libraries, and every refusal is a mcpp::warning. On macOS the member supplies the runner named `app` (macapp-run) and declares xim:macapp-run 0.1.0 with when = "run". --format dmg stages the bundle beside an Applications link and runs hdiutil create -format UDZO; it is refused on iOS. tests/app-framework-consumer: a program whose dependency is shared and which exits 7. check-apple-plan.sh passes locally against the new member and fails against 0.9.3 ("no framework step for the staged dylib"); check-apple-bundle.sh is the macOS half (load command, codesign --verify --deep --strict, the program with and without its framework, mcpp run --format app with and without --runner app, hdiutil verify and attach). --- dist/apple.cppm | 365 ++++++++++++++---- mcpp.toml | 15 +- tests/app-framework-consumer/build.mcpp | 13 + .../check-apple-bundle.sh | 149 +++++++ .../check-apple-plan.sh | 168 ++++++++ .../dep/include/app_framework_dep.h | 3 + tests/app-framework-consumer/dep/mcpp.toml | 19 + tests/app-framework-consumer/dep/src/dep.cpp | 3 + tests/app-framework-consumer/mcpp.toml | 37 ++ .../app-framework-consumer/share/greeting.txt | 1 + tests/app-framework-consumer/src/main.cpp | 11 + 11 files changed, 700 insertions(+), 84 deletions(-) create mode 100644 tests/app-framework-consumer/build.mcpp create mode 100755 tests/app-framework-consumer/check-apple-bundle.sh create mode 100755 tests/app-framework-consumer/check-apple-plan.sh create mode 100644 tests/app-framework-consumer/dep/include/app_framework_dep.h create mode 100644 tests/app-framework-consumer/dep/mcpp.toml create mode 100644 tests/app-framework-consumer/dep/src/dep.cpp create mode 100644 tests/app-framework-consumer/mcpp.toml create mode 100644 tests/app-framework-consumer/share/greeting.txt create mode 100644 tests/app-framework-consumer/src/main.cpp diff --git a/dist/apple.cppm b/dist/apple.cppm index 1d57622..210fbf1 100644 --- a/dist/apple.cppm +++ b/dist/apple.cppm @@ -43,34 +43,39 @@ // and no `options::tool` the way `wix` does in `dist/wix.cppm` -- there is // exactly one `ditto`, at a fixed path, on every Mac this can run on. // -// HOW MANY ACTIONS, AND WHY EACH IS SEPARATE. Up to five: +// HOW MANY ACTIONS, AND WHY EACH IS SEPARATE: // // 1. install `Info.plist` (always) -// 2. lay out the staged tree (always) -// 3. install the icon (only when `options::icon` is set) -// 4. codesign the bundle (only when `options::identity` is set) -// 5. the bundle itself (always, last) +// 2. lay out the program (always) +// 3. one per deployed resource (one per entry the staged tree carries) +// 4. one per closure dylib (copied to the framework directory, signed) +// 5. install the icon (only when `options::icon` is set) +// 6. codesign the bundle (always on macOS; on an iOS device row only +// when `options::identity` is set) +// 7. the bundle itself (always, last) +// 8. two for `--format dmg` (stage the bundle beside an `Applications` +// link, then `hdiutil create`) // -// 1 and 3 are separate from 2 because they have different INPUTS: `Info.plist` -// is regenerated whenever package metadata changes, the icon only when the -// project's icon file changes, and the staged tree only when the program or -// its closure changes. One action for all three would make every one of those -// changes re-run the multi-hundred-megabyte copy. 4 is last of the CONTENT -// steps and depends on the OUTPUTS of whichever of 1 to 3 actually ran, -// because a code signature covers the bundle's content at signing time -- -// signing before the content is in place is either a failure (an incomplete -// bundle) or a signature that the next file added invalidates. +// 1, 3 and 5 are separate from 2 because they have different INPUTS: +// `Info.plist` is regenerated whenever package metadata changes, the icon only +// when the project's icon file changes, and the program only when it is +// relinked. One action for all of them would make every one of those changes +// re-run every copy. 6 is last of the CONTENT steps and depends on the OUTPUTS +// of every step before it, because a code signature covers the bundle's content +// at signing time -- signing before the content is in place is either a failure +// (an incomplete bundle) or a signature that the next file added invalidates. // -// 5 EXISTS BECAUSE 1 THROUGH 4 ARE PARALLEL, AND `mcpp run` NEEDS ONE +// 7 EXISTS BECAUSE THE CONTENT STEPS ARE PARALLEL, AND `mcpp run` NEEDS ONE // OPERAND. Each of them writes a file inside the bundle and consumes none of -// the others' outputs, so a request that submits only this plan has as many -// terminal artifacts (outputs nothing else consumes) as steps actually ran -- -// up to four, never one. `mcpp run --format app` resolves to THE terminal +// the others' outputs, so a request that submits only those has as many +// terminal artifacts (outputs nothing else consumes) as steps actually ran, +// never one. `mcpp run --format app` resolves to THE terminal // artifact, so a plan with more than one has none it can hand the runner. -// Step 5's output is the bundle DIRECTORY -- the actual distributable of this +// Step 7's output is the bundle DIRECTORY -- the actual distributable of this // format -- and its inputs are every other step's output, so it is always // the plan's sole terminal, in both the `Contents/`-shaped and flat-iOS -// layouts. +// layouts. Under `--format dmg` the bundle is an input of the image, and the +// `.dmg` is the terminal instead. // // `Info.plist` IS WRITTEN AT PLAN TIME, BUT NOT DIRECTLY TO ITS FINAL PATH, // AND THE DIFFERENCE MATTERS. It is configuration, so `write_if_different` @@ -85,15 +90,47 @@ // [artifact] in place, reported as up to date." Declaring the plan-time file // as this action's input is what makes a version bump reach the bundle. // -// CODESIGN IS OFF BY DEFAULT. An unsigned `.app` builds and runs locally on -// the machine that built it; a member that signed by default would fail -// every build on a machine with no identity in its keychain, which is most -// of them. Notarisation is out of scope entirely, and not merely deferred: -// it requires uploading the bundle to Apple over the network and waiting on -// a ticket, and a build must not reach the network -- the same rule -// `dist/appimage.cppm` states for appimagetool's runtime-stub download, -// here applying to a step this member does not attempt at all rather than -// one it works around. +// ON macOS THE BUNDLE IS SIGNED AD HOC UNLESS AN IDENTITY IS GIVEN. A bundle +// that carries a framework and no signature fails `codesign --verify --deep +// --strict` ("code has no resources but signature indicates they must be +// present", measured on macos-15, mcpp#635 run 2): the program the linker +// signed is sealed, and the bundle around it is not. Signing ad hoc needs no +// identity and no keychain, so it is the default, dylibs first and the bundle +// second (the same run measured that order verifying). `options::identity` +// signs with that identity instead, with `--timestamp`. Notarisation is out of +// scope entirely, and not merely deferred: it requires uploading the bundle to +// Apple over the network and waiting on a ticket, and a build must not reach +// the network -- the same rule `dist/appimage.cppm` states for appimagetool's +// runtime-stub download, here applying to a step this member does not attempt. +// +// THE CLOSURE'S DYLIBS GO TO THE FRAMEWORK DIRECTORY (mcpp 2026.9.14.2+). The +// engine reads a Mach-O program's closure and stages the dylibs it resolves +// beside the program in `bin/`, naming each in the stage manifest's `needs` +// lines. This member copies those into `Contents/Frameworks/` (`Frameworks/` on +// iOS), keeps them out of the resource directory, and gives the program the +// rpath that finds them there -- `@executable_path/../Frameworks`, or +// `@executable_path/Frameworks` on iOS -- through `mcpp::link_flag` at link +// time, so no file is edited after it is linked and no load command is +// rewritten. An rpath edit after the link was measured and not taken: it fails +// on a program linked without header padding ("larger updated load commands do +// not fit") and invalidates the linker's signature (mcpp#635 run 4). The rpath +// is added to every link of a macOS or iOS program whose build program calls +// `generate()`, packed or not, because the link happens before the pass that +// learns `--format`. +// +// `mcpp run --format app` REACHES THE BUNDLE THROUGH `macapp-run` ON macOS. +// This member supplies the runner named `app` (`xim:macapp-run`, which executes +// the bundle's `CFBundleExecutable` in the foreground, so its output and exit +// status are the program's), and mcpp uses the runner named after a format for +// that format (mcpp 2026.9.14.2+). A project that declares its own +// `[target..runners] app` keeps it. The iOS rows keep the runner their +// manifests name (`simctl-run`). +// +// `--format dmg` IS A DISK IMAGE OF THE BUNDLE. The bundle and a link to +// `/Applications` are staged in one directory and `hdiutil create -format UDZO` +// writes the image, the layout a user drags from. Measured on macos-15 +// (mcpp#635 run 2): the image is created, `hdiutil verify` accepts it, and it +// attaches read-only with the bundle and the link. macOS only. // // iOS IS THE SAME SHAPE, A FLAT LAYOUT INSTEAD OF `Contents/`, AND THREE // KEYS `-format app` NEVER WROTE (#622 B1). The target row and the SDK @@ -122,8 +159,9 @@ // unsigned -- `simctl install` does not check a signature -- so // `options::identity` is ignored there rather than attempted and left to // fail inside `codesign`, which cannot produce a device-shaped signature -// for a simulator binary in any case. The device row's `codesign` step is -// byte-for-byte the macOS one: same argv shape, same opt-in, same +// for a simulator binary in any case. The device row signs only with an +// identity, because a device refuses an ad-hoc signature; its `codesign` +// step is the macOS one with an identity: same argv shape, same // hardened-runtime and entitlements flags. // // ICONS TAKE A DIRECTORY ON THIS ROW, A FILE ON THE OTHER. macOS names one @@ -217,11 +255,10 @@ struct options { std::string minimum_system_version; // A `codesign` identity -- a name or hash `security find-identity` would - // list. Empty means unsigned, which is the default; see the header - // comment for why signing is opt-in rather than automatic. Ignored on - // the iOS Simulator row regardless of this value -- see the header - // comment's signing paragraph -- and a non-empty value there produces a - // `mcpp::warning` naming why rather than a signature. + // list. Empty means an ad-hoc signature on macOS and no signature on the + // iOS rows; see the header comment's signing paragraphs. Ignored on the + // iOS Simulator row regardless of this value, and a non-empty value there + // produces a `mcpp::warning` naming why rather than a signature. std::string identity; // `--options runtime`, the hardened runtime, only meaningful together @@ -235,6 +272,12 @@ struct options { // Where the produced bundle lands. Empty means `/.app`. std::string output; + + // `--format dmg`: the volume's name, and where the image lands. Empty + // means `app_name` and `/.dmg`. + std::string volume_name; + std::string dmg; + std::string out_dir = std::string(mcpp::out_dir()); }; @@ -267,6 +310,7 @@ struct plan { bool applies = false; std::string reason; std::string bundle_path; // the .app directory + std::string dmg_path; // the .dmg, under --format dmg std::string appdir; // pack_stage_dir(), kept for the floor check std::vector steps; explicit operator bool() const { return applies; } @@ -298,6 +342,40 @@ inline bool write_if_different(const std::filesystem::path& path, return static_cast(out); } +// Records a refusal on stderr and as a `mcpp::warning`. A member that refuses +// submits no action and its build program exits 0, and the engine discards the +// output of a build program that succeeded; its own error, "no action claimed +// --format 'app'", names no reason. The warning channel is one line per +// directive, so line breaks are folded into spaces. +inline plan& refuse(plan& p, std::string reason, const std::string& message) { + std::cerr << message << '\n'; + std::string folded; + folded.reserve(message.size()); + bool space = false; + for (std::size_t i = 0; i < message.size(); ++i) { + const char c = message[i]; + if (c == '\n' || c == '\r') { space = true; continue; } + if (space) { + if (c == ' ') continue; + folded += ' '; + space = false; + } + folded += c; + } + mcpp::warning(folded.c_str()); + p.reason = std::move(reason); + return p; +} + +// A helper script, written into `/dist-apple/` at plan time when its +// bytes differ, and run by `/bin/sh` so that no permission bit is needed. +inline std::string helper_script(const std::string& out_dir, const char* name, + std::string_view body) { + const auto path = std::filesystem::path(out_dir) / "dist-apple" / name; + write_if_different(path, body); + return path.string(); +} + inline std::string target_for(const options& opt) { if (!opt.target.empty()) return opt.target; const char* n = mcpp::package_name(); @@ -483,12 +561,13 @@ inline plan plan_for(options opt = {}) { // `pack_format()` is what says so -- see `generate` for why the // DECLARATION must not be gated the same way. const std::string requested = mcpp::pack_format(); - if (requested != "app") { + if (requested != "app" && requested != "dmg") { p.reason = requested.empty() ? "this build is not packaging" - : std::format("--format {} was requested, not app", requested); + : std::format("--format {} was requested, not app or dmg", requested); return p; } + const bool dmg = requested == "dmg"; // macOS or iOS only, and this is a refusal rather than a silent skip: a // user who typed `--format app` on Linux asked for something that does @@ -497,13 +576,16 @@ inline plan plan_for(options opt = {}) { const std::string os = mcpp::target_os(); const bool isIos = (os == "ios"); if (os != "macos" && !isIos) { - std::cerr << std::format( + return refuse(p, "not a macOS or iOS target", std::format( "mcpp.dist.apple: a .app bundle is a macOS or iOS format, and " "this build targets '{}'.\n" " use: --format tar, or build for a macOS or iOS target", - os.empty() ? "unknown" : os) << '\n'; - p.reason = "not a macOS or iOS target"; - return p; + os.empty() ? "unknown" : os)); + } + if (dmg && isIos) { + return refuse(p, "a disk image on an iOS target", + "mcpp.dist.apple: a .dmg is a macOS disk image, and this build " + "targets iOS. Use --format app for an iOS bundle."); } // `aarch64-ios-sim` and `aarch64-ios` share one OS and diverge only in // `env` (triple.cppm's own words: "the simulator is deliberately not a @@ -537,23 +619,19 @@ inline plan plan_for(options opt = {}) { const std::string target = target_for(opt); if (target.empty()) { - std::cerr << "mcpp.dist.apple: no target to bundle. Set " - "`options::target` to the program target's name.\n"; - p.reason = "no target"; - return p; + return refuse(p, "no target", "mcpp.dist.apple: no target to bundle. Set " + "`options::target` to the program target's name."); } const std::string launcher = stage.empty() ? std::format("${{mcpp.target_file:{}}}", target) : launcher_in(stage, target); if (launcher.empty()) { - std::cerr << std::format( + return refuse(p, "no launcher in the staged tree", std::format( "mcpp.dist.apple: the staged tree at {0} carries no launcher for " "target '{1}'.\n" " expected one of: {0}/{1}, {0}/bin/{1}, {0}/run.sh", - stage, target) << '\n'; - p.reason = "no launcher in the staged tree"; - return p; + stage, target)); } // CFBundleExecutable is a bare filename (see the note above). With no // staged tree the launcher is a PLACEHOLDER the engine expands later, so @@ -600,14 +678,12 @@ inline plan plan_for(options opt = {}) { if (isIos) { std::error_code ec; if (!std::filesystem::is_directory(opt.icon, ec)) { - std::cerr << std::format( + return refuse(p, "icon is not a directory", std::format( "mcpp.dist.apple: `options::icon` ({}) is not a " "directory. Set it to a directory of flat PNGs " "(one per size Apple's Home Screen and Settings need); " "this member lists their stems under `CFBundleIcons` " - "and does not generate sizes itself.", opt.icon) << '\n'; - p.reason = "icon is not a directory"; - return p; + "and does not generate sizes itself.", opt.icon)); } for (auto const& e : std::filesystem::directory_iterator(opt.icon, ec)) { if (ec) break; @@ -616,35 +692,27 @@ inline plan plan_for(options opt = {}) { } std::sort(iosIconStems.begin(), iosIconStems.end()); if (iosIconStems.empty()) { - std::cerr << std::format( + return refuse(p, "icon directory carries no PNGs", std::format( "mcpp.dist.apple: `options::icon` ({}) carries no " - "*.png files.", opt.icon) << '\n'; - p.reason = "icon directory carries no PNGs"; - return p; + "*.png files.", opt.icon)); } } else if (!is_file(opt.icon)) { - std::cerr << std::format( + return refuse(p, "icon not found", std::format( "mcpp.dist.apple: `options::icon` ({}) was not found", - opt.icon) << '\n'; - p.reason = "icon not found"; - return p; + opt.icon)); } } if (!opt.entitlements.empty() && !is_file(opt.entitlements)) { - std::cerr << std::format( - "mcpp.dist.apple: the entitlements file {} was not found", opt.entitlements) << '\n'; - p.reason = "entitlements not found"; - return p; + return refuse(p, "entitlements not found", std::format( + "mcpp.dist.apple: the entitlements file {} was not found", opt.entitlements)); } const char* pv = mcpp::package_version(); const std::string version = !opt.version.empty() ? opt.version : (pv && *pv ? std::string(pv) : std::string()); if (version.empty()) { - std::cerr << "mcpp.dist.apple: no version to state. Set " - "`[package] version` or `options::version`.\n"; - p.reason = "no version"; - return p; + return refuse(p, "no version", "mcpp.dist.apple: no version to state. Set " + "`[package] version` or `options::version`."); } const std::string name = app_name_for(opt); @@ -674,14 +742,44 @@ inline plan plan_for(options opt = {}) { macIconName, iosIconStems); const std::string plistSrc = (std::filesystem::path(opt.out_dir) / (name + "-Info.plist")).string(); if (!write_if_different(plistSrc, plistBytes)) { - std::cerr << std::format("mcpp.dist.apple: cannot write {}", plistSrc) << '\n'; - p.reason = "cannot write Info.plist"; - return p; + return refuse(p, "cannot write Info.plist", std::format("mcpp.dist.apple: cannot write {}", plistSrc)); } p.bundle_path = bundlePath; p.appdir = stage; + // THE CLOSURE THE ENGINE STAGED (mcpp 2026.9.14.2+). Every `needs` line + // whose staged path is under `bin/` is a dylib the engine placed beside + // the program; it goes to the framework directory and not to the + // resources. An incomplete closure is reported, not refused: a bundle + // without one of its libraries is the project's to judge, and the message + // names what is missing. + std::vector frameworks; // staged paths relative to `bin/` + if (!stage.empty()) { + const auto staged = mcpp::plugins::stage::read_manifest(stage); + for (auto const& n : staged.needs) + if (n.where.size() > 4 && n.where.starts_with("bin/")) + frameworks.push_back(n.where.substr(4)); + std::ranges::sort(frameworks); + frameworks.erase(std::unique(frameworks.begin(), frameworks.end()), frameworks.end()); + if (staged.found && !staged.walked) { + std::string names; + for (auto const& n : staged.needs) + if (n.where == "unresolved") names += (names.empty() ? "" : ", ") + n.name; + mcpp::warning(std::format( + "mcpp.dist.apple: the program's closure is incomplete{}, so the " + "bundle does not carry every library the program loads: {}", + names.empty() ? std::string() : " (" + names + ")", + staged.reason.empty() ? std::string("no reason was given") : staged.reason).c_str()); + } + } + const std::string frameworksDir = isIos ? bundlePath + "/Frameworks" + : bundlePath + "/Contents/Frameworks"; + // WHO SIGNS WHAT. macOS signs every bundle, ad hoc without an identity; + // an iOS device row signs only with one; the simulator row never signs. + const bool signs = !isSim && (!isIos || !opt.identity.empty()); + const std::string signingIdentity = opt.identity.empty() ? std::string("-") : opt.identity; + // Every action's output that later steps may need to depend on, gathered // as they are declared so the final, conditional codesign step can name // exactly the ones that ran. @@ -764,6 +862,11 @@ inline plan plan_for(options opt = {}) { std::ranges::sort(entries); for (auto const& e : entries) { const std::string rel = e.filename().string(); + // A dylib of the closure is a framework, not a resource. A + // directory is copied whole, so a dylib staged inside one (an + // `@executable_path//` install name) is copied to the + // framework directory as well. + if (std::ranges::find(frameworks, rel) != frameworks.end()) continue; step res; res.id = "mcpp.dist.apple.resource." + rel; res.role = "artifact"; @@ -776,6 +879,42 @@ inline plan plan_for(options opt = {}) { } } + // THE FRAMEWORKS: each copied from the staged tree and, when the bundle is + // signed, signed before the bundle is -- a bundle signature seals nested + // code that is already signed. + if (!frameworks.empty()) { + const std::string copyFramework = helper_script(opt.out_dir, "copy-framework.sh", + "#!/bin/sh\n" + "# mcpp.dist.apple helper. Do not edit.\n" + "# copy-framework.sh sign|nosign [identity]\n" + "set -e\n" + "src=\"$1\"; dst=\"$2\"; mode=\"$3\"; identity=\"$4\"\n" + "mkdir -p \"$(dirname \"$dst\")\"\n" + "ditto \"$src\" \"$dst\"\n" + "if [ \"$mode\" = sign ]; then\n" + " if [ \"$identity\" = - ]; then\n" + " codesign --force --sign - \"$dst\"\n" + " else\n" + " codesign --force --sign \"$identity\" --timestamp \"$dst\"\n" + " fi\n" + "fi\n"); + for (auto const& rel : frameworks) { + const std::string source = (std::filesystem::path(stage) / "bin" / rel).string(); + const std::string dest = frameworksDir + "/" + rel; + step fw; + fw.id = "mcpp.dist.apple.framework." + rel; + fw.role = "artifact"; + fw.description = "APP FRAMEWORK " + rel; + fw.argv = { "/bin/sh", copyFramework, source, dest }; + if (signs) { fw.argv.push_back("sign"); fw.argv.push_back(signingIdentity); } + else { fw.argv.push_back("nosign"); } + fw.inputs = { source, "${mcpp.stage_dir}" }; + fw.outputs = { dest }; + p.steps.push_back(fw); + assembled.push_back(dest); + } + } + if (!opt.icon.empty()) { step icon; icon.id = "mcpp.dist.apple.icon"; @@ -812,22 +951,30 @@ inline plan plan_for(options opt = {}) { "Simulator row -- simulator bundles install unsigned, and " "codesign cannot produce a device-shaped signature for one. Set " "identity for a device build (aarch64-ios) instead."); - } else if (!opt.identity.empty()) { + } else if (signs) { step sign; sign.id = "mcpp.dist.apple.codesign"; sign.role = "artifact"; - sign.description = "CODESIGN"; - sign.argv = { "codesign", "--force", "--sign", opt.identity, "--timestamp" }; - if (opt.hardened_runtime) { sign.argv.push_back("--options"); sign.argv.push_back("runtime"); } - if (!opt.entitlements.empty()) { - sign.argv.push_back("--entitlements"); - sign.argv.push_back(opt.entitlements); + sign.description = opt.identity.empty() ? "CODESIGN (AD HOC)" : "CODESIGN"; + // AD HOC TAKES NO TIMESTAMP AND NO RUNTIME OPTIONS: a timestamp is a + // statement by Apple's service about an identity, and the hardened + // runtime and entitlements are this option set's, which states them + // together with an identity. + sign.argv = { "codesign", "--force", "--sign", signingIdentity }; + if (!opt.identity.empty()) { + sign.argv.push_back("--timestamp"); + if (opt.hardened_runtime) { sign.argv.push_back("--options"); sign.argv.push_back("runtime"); } + if (!opt.entitlements.empty()) { + sign.argv.push_back("--entitlements"); + sign.argv.push_back(opt.entitlements); + } } sign.argv.push_back(bundlePath); // Depends on every other step's output, because codesign covers the // bundle's content at signing time -- see the header comment. sign.inputs = assembled; - if (!opt.entitlements.empty()) sign.inputs.push_back(opt.entitlements); + if (!opt.identity.empty() && !opt.entitlements.empty()) + sign.inputs.push_back(opt.entitlements); // codesign has no flag to write a receipt to an arbitrary path, so // this names the one file signing a BUNDLE (rather than a flat // Mach-O) is documented to write as part of embedding the signature: @@ -870,6 +1017,48 @@ inline plan plan_for(options opt = {}) { bundle.outputs = { bundlePath }; p.steps.push_back(bundle); + // `--format dmg`: the bundle beside an `Applications` link, then the image. + // The staging directory is emptied and refilled by its own step, so a file + // removed from the bundle does not survive into the next image; `-ov` + // replaces an image a previous pack wrote. + if (dmg) { + const std::string volume = !opt.volume_name.empty() ? opt.volume_name : name; + const std::string dmgPath = !opt.dmg.empty() ? opt.dmg + : (std::filesystem::path(opt.out_dir) / (name + ".dmg")).string(); + const std::string dmgStage = (std::filesystem::path(opt.out_dir) / "dist-apple" / "dmg").string(); + const std::string stageDmg = helper_script(opt.out_dir, "stage-dmg.sh", + "#!/bin/sh\n" + "# mcpp.dist.apple helper. Do not edit.\n" + "# stage-dmg.sh \n" + "set -e\n" + "bundle=\"$1\"; stage=\"$2\"; name=\"$3\"\n" + "rm -rf \"$stage\"\n" + "mkdir -p \"$stage\"\n" + "ditto \"$bundle\" \"$stage/$name\"\n" + "ln -s /Applications \"$stage/Applications\"\n"); + + step staging; + staging.id = "mcpp.dist.apple.dmg-stage"; + staging.role = "artifact"; + staging.description = "DMG STAGE"; + staging.argv = { "/bin/sh", stageDmg, bundlePath, dmgStage, + std::filesystem::path(bundlePath).filename().string() }; + staging.inputs = { bundlePath }; + staging.outputs = { dmgStage }; + p.steps.push_back(staging); + + step image; + image.id = "mcpp.dist.apple.dmg"; + image.role = "artifact"; + image.description = "HDIUTIL CREATE"; + image.argv = { "hdiutil", "create", "-volname", volume, "-srcfolder", dmgStage, + "-format", "UDZO", "-ov", dmgPath }; + image.inputs = { dmgStage }; + image.outputs = { dmgPath }; + p.steps.push_back(image); + p.dmg_path = dmgPath; + } + p.applies = true; return p; } @@ -948,8 +1137,22 @@ inline bool submit(const plan& p) { // collected from a pass that asked for nothing. A member that declared only // when asked still works for its author -- they always pass their own // format -- and makes the set unknowable for everyone else. +// +// THE FRAMEWORK RPATH AND THE `app` RUNNER ARE DECLARED ON EVERY PASS, TOO. The +// program is linked before the pass that learns `--format`, so the rpath that +// finds `Contents/Frameworks/` has to reach every link; and `mcpp run --format +// app` looks the runner up in the pass that runs the bundle, which is not a +// packaging pass. inline bool generate(options opt = {}) { mcpp::provides_pack_format("app"); + mcpp::provides_pack_format("dmg"); + const std::string os = mcpp::target_os(); + if (os == "macos") { + mcpp::link_flag("-Wl,-rpath,@executable_path/../Frameworks"); + mcpp::runner("app", "macapp-run"); + } else if (os == "ios") { + mcpp::link_flag("-Wl,-rpath,@executable_path/Frameworks"); + } return submit(plan_for(std::move(opt))); } diff --git a/mcpp.toml b/mcpp.toml index 462db7a..4963b1e 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -391,9 +391,18 @@ implies = ["surface"] [target.windows.feature-xlings.dist-wix] "xim:wix" = "5.0.2" -# `dist-apple` declares no payload for `codesign`: it is part of Xcode, which is -# not redistributable, so the member locates it and says where it looked. The -# open-source `rcodesign` is the package that would replace that lookup. +# `dist-apple` declares no payload for `codesign`, `ditto` or `hdiutil`: they are +# part of macOS and Xcode, which are not redistributable, so the member runs the +# system's. The open-source `rcodesign` is the package that would replace the +# `codesign` lookup. +# +# THE RUNNER IS A PAYLOAD, AND ONLY `mcpp run` NEEDS IT. `mcpp run --format app` +# on macOS reaches the bundle through the runner named `app` this member +# supplies, `macapp-run`, which executes the bundle's executable in the +# foreground so that its output and exit status are the program's. `when = +# "run"` keeps a build or a pack from installing it. +[target.'cfg(os = "macos")'.feature-xlings.dist-apple] +"xim:macapp-run" = { version = "0.1.0", when = "run" } # ── The environment `dist-apk` needs ─────────────────────────────────────── # diff --git a/tests/app-framework-consumer/build.mcpp b/tests/app-framework-consumer/build.mcpp new file mode 100644 index 0000000..e1ae390 --- /dev/null +++ b/tests/app-framework-consumer/build.mcpp @@ -0,0 +1,13 @@ +// No `identity`: the bundle is signed ad hoc on macOS, and the fixture +// declares no runner, so `mcpp run --format app` reaches the bundle through the +// runner `dist-apple` supplies. +import std; +import mcpp; +import mcpp.dist.apple; + +int main() { + mcpp::dist::apple::options opt; + opt.target = "app-framework-consumer"; + opt.app_name = "AppFrameworkConsumer"; + return mcpp::dist::apple::generate(opt) ? 0 : 1; +} diff --git a/tests/app-framework-consumer/check-apple-bundle.sh b/tests/app-framework-consumer/check-apple-bundle.sh new file mode 100755 index 0000000..3d7ea8a --- /dev/null +++ b/tests/app-framework-consumer/check-apple-bundle.sh @@ -0,0 +1,149 @@ +#!/usr/bin/env bash +# dist-apple on macOS, measured on the bundle and the image (mcpp#634, B1 to B3). +# +# The plan-level half is `check-apple-plan.sh`; this half needs the tools only +# a Mac has (`otool`, `codesign`, `hdiutil`) and runs the program. +# +# 1. The program is linked with the rpath `@executable_path/../Frameworks`, +# as written: the engine anchors no rpath that begins with a loader token +# (mcpp 2026.9.14.2+), and the member adds it through `mcpp::link_flag`. +# 2. The bundle carries the dependency's dylib in `Contents/Frameworks/` and +# not among the resources, keeps the deployed file under +# `Contents/Resources/`, and is signed ad hoc so that `codesign --verify +# --deep --strict` accepts it. +# 3. The bundled program loads the framework copy and exits 7 with its +# output; a copy of the bundle without the framework stops with "Library +# not loaded". +# 4. `mcpp run --format app`, with no runner in the manifest, runs the bundle +# through `macapp-run` and returns the program's 7 with its output and +# arguments; `--runner app` does the same. +# 5. `mcpp pack --format dmg` writes an image `hdiutil verify` accepts, which +# attaches read-only holding the bundle and an `Applications` link. +# +# Usage: MCPP= ./check-apple-bundle.sh (run from this directory) +set -eu + +MCPP="${MCPP:-mcpp}" +NAME=AppFrameworkConsumer +EXE=app-framework-consumer +DYLIB=libapp-framework-dep.dylib +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +reading() { printf 'READING %s: %s\n' "$1" "$2"; } +packed() { sed -n 's/^ *Packed //p' "$1" | tail -1; } +work=$(mktemp -d) + +# ── 1. the rpath, at link time ───────────────────────────────────────────── +echo "== 1. the program's rpath ==" +rm -rf target +"$MCPP" build > bundle-build.log 2>&1 || fail "mcpp build failed" bundle-build.log +if find target -name '*.app' | grep -q .; then fail "a plain build produced a bundle"; fi +programs=$(find target -type f -name "$EXE" ! -path '*/.build-mcpp/*') +[ "$(printf '%s\n' "$programs" | grep -c .)" = 1 ] \ + || fail "expected one linked $EXE under target/, found: $(echo $programs)" bundle-build.log +otool -l "$programs" > bundle-loadcommands.log +rpaths=$(awk '/cmd LC_RPATH/ { getline; getline; print $2 }' bundle-loadcommands.log) +reading rpaths "$(echo $rpaths)" +printf '%s\n' "$rpaths" | grep -qxF '@executable_path/../Frameworks' \ + || fail "the program has no LC_RPATH @executable_path/../Frameworks" bundle-loadcommands.log +if printf '%s\n' "$rpaths" | grep -q '.@executable_path'; then + fail "an rpath carries @executable_path after a directory, anchored" bundle-loadcommands.log +fi +otool -L "$programs" > bundle-needed.log +grep -q "@rpath/$DYLIB" bundle-needed.log || fail "the program does not load @rpath/$DYLIB" bundle-needed.log +echo "ok: LC_RPATH @executable_path/../Frameworks, as written, and the dependency is @rpath/$DYLIB" + +# ── 2. the bundle ────────────────────────────────────────────────────────── +echo "== 2. the bundle carries the framework and is signed ad hoc ==" +"$MCPP" pack --format app > bundle-pack.log 2>&1 || fail "mcpp pack --format app failed" bundle-pack.log +app=$(packed bundle-pack.log) +case "$app" in *"/$NAME.app") ;; *) fail "the pack reported '$app', not $NAME.app" bundle-pack.log ;; esac +[ -d "$app" ] || fail "the reported bundle $app is not a directory" bundle-pack.log +[ -f "$app/Contents/Frameworks/$DYLIB" ] || { find "$app" | sort; fail "no Contents/Frameworks/$DYLIB"; } +if find "$app/Contents/Resources" "$app/Contents/MacOS" -name '*.dylib' 2>/dev/null | grep -q .; then + find "$app" | sort; fail "a dylib is under Contents/Resources or Contents/MacOS" +fi +[ -f "$app/Contents/Resources/data/greeting.txt" ] || { find "$app" | sort; fail "the deployed file is not under Contents/Resources"; } +codesign --verify --deep --strict --verbose=2 "$app" > bundle-codesign.log 2>&1 \ + || fail "codesign --verify --deep --strict refused the bundle" bundle-codesign.log +reading codesign-verify "$(tr '\n' ' ' < bundle-codesign.log)" +# A linked arm64 program is already signed ad hoc by the linker, so the bundle's +# own signature is read from its sealed resources, and the framework's from +# the absence of the linker's flag. +codesign -dv "$app" > bundle-signature.log 2>&1 || true +reading bundle-signature "$(grep -E '^(Signature|CodeDirectory|Sealed Resources)' bundle-signature.log | tr '\n' ' ')" +grep -q '^Signature=adhoc' bundle-signature.log || fail "the bundle's signature is not ad hoc" bundle-signature.log +grep -q '^Sealed Resources' bundle-signature.log || fail "the bundle's resources are not sealed" bundle-signature.log +codesign -dv "$app/Contents/Frameworks/$DYLIB" > bundle-framework-signature.log 2>&1 || true +reading framework-signature "$(grep -E '^(Signature|CodeDirectory)' bundle-framework-signature.log | tr '\n' ' ')" +grep -q '^Signature=adhoc' bundle-framework-signature.log \ + || fail "the framework is not signed" bundle-framework-signature.log +if grep -q 'linker-signed' bundle-framework-signature.log; then + fail "the framework carries the linker's signature, not one of its own" bundle-framework-signature.log +fi +echo "ok: Contents/Frameworks/$DYLIB, nothing loadable among the resources, and codesign --verify --deep --strict passes" + +# ── 3. the bundled program ───────────────────────────────────────────────── +echo "== 3. the bundled program loads the framework ==" +rc=0 +DYLD_PRINT_LIBRARIES=1 "$app/Contents/MacOS/$EXE" > bundle-run.out 2> bundle-run.err || rc=$? +reading bundle-run "exit=$rc $(cat bundle-run.out)" +[ "$rc" -eq 7 ] || fail "the bundled program exited $rc, not 7" bundle-run.out bundle-run.err +grep -qx 'framework-1-2-3 argc=1' bundle-run.out || fail "the bundled program's output is missing" bundle-run.out +loaded=$(grep "$DYLIB" bundle-run.err | head -1) +reading loaded "$loaded" +case "$loaded" in *"/$NAME.app/Contents/Frameworks/$DYLIB") ;; + *) fail "the program did not load the framework copy" bundle-run.err ;; esac +cp -R "$app" "$work/" +rm "$work/$NAME.app/Contents/Frameworks/$DYLIB" +rc=0 +"$work/$NAME.app/Contents/MacOS/$EXE" > bundle-noframework.out 2>&1 || rc=$? +reading without-framework "exit=$rc $(head -2 bundle-noframework.out | tr '\n' ' ')" +[ "$rc" -ne 0 ] || fail "the program ran without its framework" bundle-noframework.out +grep -q 'Library not loaded' bundle-noframework.out || fail "the failure is not 'Library not loaded'" bundle-noframework.out +echo "ok: exit 7 through the framework copy; without it, exit $rc and 'Library not loaded'" + +# ── 4. mcpp run --format app ─────────────────────────────────────────────── +echo "== 4. mcpp run --format app, through the runner dist-apple supplies ==" +if grep -q 'runner' mcpp.toml; then fail "the fixture's manifest names a runner" mcpp.toml; fi +# mcpp_run +mcpp_run() { + local variant="$1"; shift + rc=0 + "$MCPP" run "$@" > "bundle-mcpp-run-$variant.log" 2>&1 || rc=$? + reading "mcpp-run-$variant" "exit=$rc $(grep -m1 'Running' "bundle-mcpp-run-$variant.log" || echo 'no Running line')" + [ "$rc" -eq 7 ] || fail "mcpp run $* exited $rc, not 7" "bundle-mcpp-run-$variant.log" + grep -q 'Running `.*macapp-run' "bundle-mcpp-run-$variant.log" \ + || fail "the status line does not name macapp-run" "bundle-mcpp-run-$variant.log" + grep -qx 'framework-1-2-3 argc=2' "bundle-mcpp-run-$variant.log" \ + || fail "the program's output with one argument is missing" "bundle-mcpp-run-$variant.log" +} +mcpp_run format --format app -- extra +mcpp_run runner --format app --runner app -- extra +echo "ok: mcpp run --format app, and with --runner app, return 7 with the program's output and argument" + +# ── 5. the disk image ────────────────────────────────────────────────────── +echo "== 5. mcpp pack --format dmg ==" +"$MCPP" pack --format dmg > bundle-dmg.log 2>&1 || fail "mcpp pack --format dmg failed" bundle-dmg.log +dmg=$(packed bundle-dmg.log) +case "$dmg" in *"/$NAME.dmg") ;; *) fail "the pack reported '$dmg', not $NAME.dmg" bundle-dmg.log ;; esac +[ -f "$dmg" ] || fail "the reported image $dmg does not exist" bundle-dmg.log +hdiutil verify "$dmg" > bundle-hdiutil-verify.log 2>&1 || fail "hdiutil verify refused the image" bundle-hdiutil-verify.log +reading hdiutil-verify "$(grep -i 'checksum' bundle-hdiutil-verify.log | tail -1)" +mnt="$work/mnt" +mkdir -p "$mnt" +hdiutil attach -nobrowse -readonly -mountpoint "$mnt" "$dmg" > bundle-attach.log 2>&1 \ + || fail "hdiutil attach failed" bundle-attach.log +detach() { hdiutil detach "$mnt" > /dev/null 2>&1 || hdiutil detach -force "$mnt" > /dev/null 2>&1 || true; } +trap detach EXIT +reading image-root "$(ls -1 "$mnt" | tr '\n' ' ')" +[ -x "$mnt/$NAME.app/Contents/MacOS/$EXE" ] || fail "the image does not hold the bundle's program" +[ -f "$mnt/$NAME.app/Contents/Frameworks/$DYLIB" ] || fail "the image's bundle lacks the framework" +[ -L "$mnt/Applications" ] || fail "the image holds no Applications link" +[ "$(readlink "$mnt/Applications")" = /Applications ] || fail "the Applications link points to $(readlink "$mnt/Applications")" +codesign --verify --deep --strict "$mnt/$NAME.app" > bundle-image-codesign.log 2>&1 \ + || fail "the bundle inside the image does not verify" bundle-image-codesign.log +detach +trap - EXIT +echo "ok: the image verifies and attaches read-only with the signed bundle and Applications -> /Applications" + +echo "PASS: dist-apple's framework, rpath, ad-hoc signature, app runner and disk image, on the bundle" diff --git a/tests/app-framework-consumer/check-apple-plan.sh b/tests/app-framework-consumer/check-apple-plan.sh new file mode 100755 index 0000000..6d22c16 --- /dev/null +++ b/tests/app-framework-consumer/check-apple-plan.sh @@ -0,0 +1,168 @@ +#!/usr/bin/env bash +# Plan-level check for dist-apple's closure, signing, runner and disk image on +# the macOS row (mcpp#634, B1 to B3). +# +# The real bundle is measured on the macOS runner. This script measures, on any +# host, what the compiled build program PLANS under the environment the engine +# sets for `mcpp pack --format app` and `--format dmg` on macOS: the build +# program's target accessors read that environment and nothing else, so a +# process given the same variables takes the branches a real pack takes. The +# staged tree and its stage manifest are fabricated in the shape the engine +# writes (mcpp's docs/50, "The stage manifest"). +# +# Usage: MCPP= ./check-apple-plan.sh (run from this directory) +set -e + +MCPP="${MCPP:-mcpp}" +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +command -v python3 >/dev/null || fail "python3 is required to read the planned actions" + +rm -rf target +"$MCPP" build > build.log 2>&1 || fail "the host build failed to compile build.mcpp" build.log +BIN=target/.build-mcpp/build.mcpp.bin +[ -x "$BIN" ] || fail "no compiled build.mcpp.bin at $BIN" build.log + +# stage_tree : a staged tree with the program, its dylib and a +# deployed resource, and the stage manifest the engine writes for it. +stage_tree() { + local stage + stage=$(mktemp -d) + mkdir -p "$stage/bin/data" + head -c 5000 /dev/urandom > "$stage/bin/app-framework-consumer" + head -c 4000 /dev/urandom > "$stage/bin/libapp-framework-dep.dylib" + echo hello > "$stage/bin/data/greeting.txt" + { + printf 'closure = %s\n' "$1" + if [ "$1" = not-walked ]; then + printf 'reason = the loader finds no file for @rpath/libmissing.dylib\n' + fi + printf 'needs\t/usr/lib/libSystem.B.dylib\tplatform\n' + printf 'needs\t@rpath/libapp-framework-dep.dylib\tbin/libapp-framework-dep.dylib\n' + if [ "$1" = not-walked ]; then + printf 'needs\t@rpath/libmissing.dylib\tunresolved\n' + fi + printf '5000 bin/app-framework-consumer\n' + printf '4000 bin/libapp-framework-dep.dylib\n' + } > "$stage.stage-manifest" + echo "$stage" +} + +# run_program +run_program() { + local out + out=$(mktemp -d) + env -i PATH="$PATH" \ + MCPP_PACK_FORMAT="$1" MCPP_TARGET_OS="$2" MCPP_TARGET_ENV="$3" \ + MCPP_PACK_STAGE_DIR="$4" MCPP_MANIFEST_DIR="$PWD" \ + MCPP_PKG_NAME=app-framework-consumer MCPP_PKG_VERSION=0.1.0 \ + MCPP_OUT_DIR="$out" \ + "$BIN" > "$5" 2>&1 || true +} + +# check action) and `lines`> +check() { + python3 - "$1" "$2" <<'PY' || exit 1 +import json, sys +log, code = sys.argv[1], sys.argv[2] +lines = open(log).read().splitlines() +actions = {} +for l in lines: + if l.startswith("mcpp:action="): + a = json.loads(l[len("mcpp:action="):]) + actions[a["id"]] = a +def fail(msg): + print("FAIL: " + msg) + print("--- " + log + " ---") + print("\n".join(lines)) + sys.exit(1) +exec(code) +PY +} + +echo "== --format app on macOS, with a staged dylib ==" +stage=$(stage_tree walked) +run_program app macos "" "$stage" /tmp/apple-plan-app.log +check /tmp/apple-plan-app.log ' +fw = actions.get("mcpp.dist.apple.framework.libapp-framework-dep.dylib") +if not fw: fail("no framework step for the staged dylib") +if not fw["outputs"] or not fw["outputs"][0].endswith("/AppFrameworkConsumer.app/Contents/Frameworks/libapp-framework-dep.dylib"): + fail("the framework step does not write Contents/Frameworks/libapp-framework-dep.dylib") +if fw["command"][-2:] != ["sign", "-"]: + fail("the framework step does not sign ad hoc: " + repr(fw["command"])) +if any(i.startswith("mcpp.dist.apple.resource.libapp") for i in actions): + fail("the staged dylib is also planned as a resource") +if "mcpp.dist.apple.resource.data" not in actions: + fail("the deployed resource is not planned") +sign = actions.get("mcpp.dist.apple.codesign") +if not sign: fail("no codesign step on macOS without an identity") +if sign["command"][:4] != ["codesign", "--force", "--sign", "-"] or "--timestamp" in sign["command"]: + fail("the bundle is not signed ad hoc: " + repr(sign["command"])) +if fw["outputs"][0] not in sign["inputs"]: + fail("the bundle signature does not follow the framework") +bundle = actions.get("mcpp.dist.apple.bundle") +if not bundle or sign["outputs"][0] not in bundle["inputs"]: + fail("the bundle step does not follow the signature") +if any(k.startswith("mcpp.dist.apple.dmg") for k in actions): + fail("--format app planned a disk image") +if "mcpp:link-flag=-Wl,-rpath,@executable_path/../Frameworks" not in lines: + fail("no framework rpath was emitted for the link") +if "mcpp:runner-named=app:macapp-run" not in lines: + fail("the app runner was not supplied") +' +echo "ok: the dylib is a framework signed ad hoc, not a resource; the bundle is signed after it; the rpath and the runner are declared" + +echo "== --format dmg on macOS ==" +run_program dmg macos "" "$stage" /tmp/apple-plan-dmg.log +check /tmp/apple-plan-dmg.log ' +bundle = actions.get("mcpp.dist.apple.bundle") +st = actions.get("mcpp.dist.apple.dmg-stage") +img = actions.get("mcpp.dist.apple.dmg") +if not (bundle and st and img): fail("the disk image steps are not planned") +if bundle["outputs"][0] not in st["inputs"]: fail("the image staging does not take the bundle") +if st["outputs"][0] not in img["inputs"]: fail("hdiutil does not take the staging directory") +cmd = img["command"] +if cmd[:2] != ["hdiutil", "create"] or "UDZO" not in cmd or not cmd[-1].endswith("/AppFrameworkConsumer.dmg"): + fail("the image command is not hdiutil create -format UDZO ... AppFrameworkConsumer.dmg: " + repr(cmd)) +consumed = set(i for a in actions.values() for i in a["inputs"]) +terminals = [a["id"] for a in actions.values() if not set(a["outputs"]) & consumed] +if terminals != ["mcpp.dist.apple.dmg"]: + fail("the image is not the sole terminal artifact: " + repr(terminals)) +' +echo "ok: the bundle is staged beside the link and the .dmg is the sole terminal artifact" + +echo "== an incomplete closure is reported ==" +stage2=$(stage_tree not-walked) +run_program app macos "" "$stage2" /tmp/apple-plan-notwalked.log +check /tmp/apple-plan-notwalked.log ' +if not any(l.startswith("mcpp:warning=") and "libmissing.dylib" in l and "closure is incomplete" in l for l in lines): + fail("the incomplete closure is not reported, naming the unresolved library") +if "mcpp.dist.apple.bundle" not in actions: + fail("an incomplete closure stopped the bundle") +' +echo "ok: an incomplete closure is a warning naming the library, and the bundle is still planned" + +echo "== --format dmg on iOS is refused ==" +run_program dmg ios sim "$stage" /tmp/apple-plan-dmg-ios.log +check /tmp/apple-plan-dmg-ios.log ' +if actions: fail("an iOS disk image planned actions") +if not any(l.startswith("mcpp:warning=") and "a .dmg is a macOS disk image" in l for l in lines): + fail("the refusal is not a warning") +' +echo "ok: an iOS disk image is refused, as a warning" + +echo "== the iOS simulator row ==" +run_program app ios sim "$stage" /tmp/apple-plan-ios.log +check /tmp/apple-plan-ios.log ' +fw = actions.get("mcpp.dist.apple.framework.libapp-framework-dep.dylib") +if not fw or not fw["outputs"][0].endswith("/AppFrameworkConsumer.app/Frameworks/libapp-framework-dep.dylib"): + fail("the simulator bundle does not carry the dylib in Frameworks/") +if fw["command"][-1] != "nosign": fail("the simulator framework is signed") +if "mcpp.dist.apple.codesign" in actions: fail("the simulator bundle is signed") +if "mcpp:link-flag=-Wl,-rpath,@executable_path/Frameworks" not in lines: + fail("no iOS framework rpath was emitted") +if any(l.startswith("mcpp:runner-named=app:") for l in lines): + fail("the macOS runner was supplied on iOS") +' +echo "ok: the simulator bundle carries Frameworks/, unsigned, with the iOS rpath and no macOS runner" + +echo "PASS: dist-apple's closure, signing, runner and disk image, at the plan level" diff --git a/tests/app-framework-consumer/dep/include/app_framework_dep.h b/tests/app-framework-consumer/dep/include/app_framework_dep.h new file mode 100644 index 0000000..a0e8a97 --- /dev/null +++ b/tests/app-framework-consumer/dep/include/app_framework_dep.h @@ -0,0 +1,3 @@ +#pragma once + +const char* app_framework_dep_marker(); diff --git a/tests/app-framework-consumer/dep/mcpp.toml b/tests/app-framework-consumer/dep/mcpp.toml new file mode 100644 index 0000000..ffceedb --- /dev/null +++ b/tests/app-framework-consumer/dep/mcpp.toml @@ -0,0 +1,19 @@ +# Fixture: the dependency `app-framework-consumer` links as a shared library, +# which `dist-apple` carries in the bundle's framework directory. +[package] +name = "app-framework-dep" +namespace = "mcpp" +version = "0.1.0" +description = "Fixture: a dependency that becomes a framework in a bundle" +license = "Apache-2.0" +authors = ["mcpp-community"] + +[language] +standard = "c++23" + +[targets.app-framework-dep] +kind = "lib" + +[build] +sources = ["src/dep.cpp"] +include_dirs = ["include"] diff --git a/tests/app-framework-consumer/dep/src/dep.cpp b/tests/app-framework-consumer/dep/src/dep.cpp new file mode 100644 index 0000000..664e650 --- /dev/null +++ b/tests/app-framework-consumer/dep/src/dep.cpp @@ -0,0 +1,3 @@ +#include "app_framework_dep.h" + +const char* app_framework_dep_marker() { return "framework-1-2-3"; } diff --git a/tests/app-framework-consumer/mcpp.toml b/tests/app-framework-consumer/mcpp.toml new file mode 100644 index 0000000..76852d7 --- /dev/null +++ b/tests/app-framework-consumer/mcpp.toml @@ -0,0 +1,37 @@ +# Fixture: a macOS program whose dependency is a SHARED library, packed as a +# bundle and as a disk image by dist-apple (mcpp#634, B1 to B3). +# +# The engine reads the program's closure and stages the dependency's dylib +# beside the program (mcpp 2026.9.14.2+); `dist-apple` then carries it in +# `Contents/Frameworks/`, gives the program the rpath that finds it there, +# signs the bundle ad hoc, supplies the runner named `app`, and writes a +# `.dmg`. The program exits 7, so that a runner which loses the program's exit +# status is visible. +# +# Built for real on the macOS runner (`rules-cross-platform`), and checked at +# the plan level on Linux by `check-apple-plan.sh`, which runs the compiled +# build program under the environment the engine sets for a macOS pack. +[package] +name = "app-framework-consumer" +version = "0.1.0" +description = "Fixture: a bundle that carries its dependency as a framework" +license = "Apache-2.0" +authors = ["mcpp-community"] + +[language] +standard = "c++23" +modules = true +import_std = true + +[build-dependencies.mcpp] +plugins = { path = "../..", features = ["dist-apple"], host-module = true } + +[dependencies] +mcpp.app-framework-dep = { path = "dep", linkage = "shared" } + +[targets.app-framework-consumer] +kind = "bin" +main = "src/main.cpp" + +[runtime] +deploy = [ { from = "share/greeting.txt", to = "data" } ] diff --git a/tests/app-framework-consumer/share/greeting.txt b/tests/app-framework-consumer/share/greeting.txt new file mode 100644 index 0000000..8b39c7c --- /dev/null +++ b/tests/app-framework-consumer/share/greeting.txt @@ -0,0 +1 @@ +hello from the resource directory diff --git a/tests/app-framework-consumer/src/main.cpp b/tests/app-framework-consumer/src/main.cpp new file mode 100644 index 0000000..a9a86f1 --- /dev/null +++ b/tests/app-framework-consumer/src/main.cpp @@ -0,0 +1,11 @@ +#include + +#include "app_framework_dep.h" + +// The marker comes from the dependency, so the program loads the dylib before +// it prints anything; the exit status is 7 so that a runner which reports its +// own status instead of the program's is visible. +int main(int argc, char**) { + std::printf("%s argc=%d\n", app_framework_dep_marker(), argc); + return 7; +} From 3881d1e50458f18a7297fca6090ff753f25d486e Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Mon, 14 Sep 2026 16:03:19 +0800 Subject: [PATCH 04/13] feat(dist-wix): --format setup, a Burn bundle that chains the MSI (mcpp#634) A second `wix build` over a bundle definition whose chain holds the MSI and whose user interface is WiX's stock bootstrapper application, loaded through WixToolset.BootstrapperApplications.wixext from the xim:wix payload (pinned to the 5.0.2-1 revision that carries it). The bundle is -.exe with an UpgradeCode of its own; options::bundle_output naming setup.exe is refused before wix runs (WIX0388). options::bundle_wxs replaces the generated definition and receives the MSI as $(Msi). Every refusal is a mcpp::warning. tests/msi-consumer/check-setup.sh builds the bundle and compares the MSI `wix burn extract` takes out of it with the one the first action wrote. --- dist/wix.cppm | 249 ++++++++++++++++++++++++------ mcpp.toml | 9 +- tests/msi-consumer/check-setup.sh | 54 +++++++ 3 files changed, 268 insertions(+), 44 deletions(-) create mode 100755 tests/msi-consumer/check-setup.sh diff --git a/dist/wix.cppm b/dist/wix.cppm index 0d7e484..d141057 100644 --- a/dist/wix.cppm +++ b/dist/wix.cppm @@ -77,6 +77,18 @@ // A host lookup therefore exists only where nothing can be shipped, and WiX is // no longer such a case. `options::tool` remains for a project that builds the // tool itself. +// +// `--format setup` IS A BURN BUNDLE THAT CHAINS THE MSI. Two actions: the MSI +// above, and a second `wix build` over a bundle definition whose `` holds +// that MSI and whose user interface is WiX's stock bootstrapper application +// (`bal:WixStandardBootstrapperApplication`). The stock application is an +// extension, `WixToolset.BootstrapperApplications.wixext`, which `xim:wix` +// carries from 5.0.2-1 and which `wix build` loads by path (`-ext`); without it +// the bundle is refused (WIX0200, measured on windows-2022 by xim-pkgindex's +// install check). The bundle is written as `-.exe` and never +// as `setup.exe`, a name `wix` refuses (WIX0388: Windows loads compatibility +// shims into an executable named like an installer). A project with its own +// bootstrapper application supplies its own bundle definition. module; #include @@ -154,6 +166,19 @@ struct options { // Where the produced file lands. Empty means // `/-.msi`. std::string output; + + // `--format setup`. `bundle_wxs` is a project-supplied bundle definition + // that wins over the generated one; it receives the MSI's path as the + // preprocessor variable `$(Msi)`. `license_url` is the stock application's + // `LicenseUrl`; empty hides the licence link. `extension` names the + // `WixToolset.BootstrapperApplications.wixext.dll` to load, and empty means + // the one `xim:wix` carries. `bundle_output` is where the bundle lands; + // empty means `/-.exe`. + std::string bundle_wxs; + std::string license_url; + std::string extension; + std::string bundle_output; + std::string out_dir = std::string(mcpp::out_dir()); }; @@ -176,6 +201,12 @@ struct plan { std::string target_name; // for the opportunistic size probe below std::vector argv; std::vector inputs; + // `--format setup` only: the bundle that chains the MSI above. Empty + // otherwise. + std::string bundle_output; + std::string bundle_wxs_path; + std::vector bundle_argv; + std::vector bundle_inputs; explicit operator bool() const { return applies; } }; @@ -205,6 +236,31 @@ inline bool write_if_different(const std::filesystem::path& path, return static_cast(out); } +// Records a refusal on stderr and as a `mcpp::warning`. A member that refuses +// submits no action and its build program exits 0, and the engine discards the +// output of a build program that succeeded; its own error, "no action claimed +// --format 'msi'", names no reason. The warning channel is one line per +// directive, so line breaks are folded into spaces. +inline plan& refuse(plan& p, std::string reason, const std::string& message) { + std::cerr << message << '\n'; + std::string folded; + folded.reserve(message.size()); + bool space = false; + for (std::size_t i = 0; i < message.size(); ++i) { + const char c = message[i]; + if (c == '\n' || c == '\r') { space = true; continue; } + if (space) { + if (c == ' ') continue; + folded += ' '; + space = false; + } + folded += c; + } + mcpp::warning(folded.c_str()); + p.reason = std::move(reason); + return p; +} + inline std::string target_for(const options& opt) { if (!opt.target.empty()) return opt.target; const char* n = mcpp::package_name(); @@ -409,6 +465,47 @@ inline std::string discover_tool(const options& opt) { return wix_payload_exe(); } +// The stock bootstrapper application's extension: what the project named, else +// the one `xim:wix` carries beside its tool (5.0.2-1 and later). +inline std::string discover_bal_extension(const options& opt) { + if (!opt.extension.empty()) return opt.extension; + const std::string dir = mcpp::xpkg_dir("xim", "wix"); + if (dir.empty()) return {}; + const auto dll = std::filesystem::path(dir) / "bal" / "wixext5" + / "WixToolset.BootstrapperApplications.wixext.dll"; + return is_file(dll.string()) ? dll.string() : std::string(); +} + +// A Burn bundle with WiX's stock bootstrapper application and one package, the +// MSI, named through `$(Msi)` for the reason `$(Executable)` is (the header's +// two substitution passes). The bundle carries its own UpgradeCode, derived +// from the product's identity and distinct from the MSI's: a bundle and the +// package it installs are two products to Windows. +inline std::string bundle_document(const std::string& name, const std::string& manufacturer, + const std::string& version, const std::string& upgrade_code, + const std::string& license_url) { + return std::format( + "\n" + "\n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + " \n" + "\n", + xml_escape(name), xml_escape(manufacturer), xml_escape(version), upgrade_code, + xml_escape(license_url)); +} + // A minimal WiX v4/v5/v6 definition: one `Package`, one `Component` carrying // the single file this member wraps, one `Feature` referencing it. WiX can // derive a stable Component GUID from the component's own target path by @@ -459,38 +556,35 @@ inline plan plan_for(options opt = {}) { // `pack_format()` is what says so -- see `generate` for why the // DECLARATION must not be gated the same way. const std::string requested = mcpp::pack_format(); - if (requested != "msi") { + if (requested != "msi" && requested != "setup") { p.reason = requested.empty() ? "this build is not packaging" - : std::format("--format {} was requested, not msi", requested); + : std::format("--format {} was requested, not msi or setup", requested); return p; } + const bool setup = requested == "setup"; // Windows only, and this is a refusal rather than a silent skip: a user // who typed `--format msi` on Linux asked for something that does not // exist there, and the engine has already accepted the value because the // graph declared it. if (const std::string os = mcpp::target_os(); os != "windows") { - std::cerr << std::format( + return refuse(p, "not a Windows target", std::format( "mcpp.dist.wix: an MSI is a Windows format, and this build targets " "'{}'.\n" " use: --format tar, or build for a Windows target", - os.empty() ? "unknown" : os) << '\n'; - p.reason = "not a Windows target"; - return p; + os.empty() ? "unknown" : os)); } const std::string target = target_for(opt); if (target.empty()) { - std::cerr << "mcpp.dist.wix: no target to package. Set " - "`options::target` to the program target's name.\n"; - p.reason = "no target"; - return p; + return refuse(p, "no target", "mcpp.dist.wix: no target to package. Set " + "`options::target` to the program target's name."); } const std::string tool = discover_tool(opt); if (tool.empty()) { - std::cerr << std::format( + return refuse(p, "wix not found", std::format( "mcpp.dist.wix: the wix CLI was not found.\n" " xpkg_dir(\"xim\", \"wix\") answered \"{}\"; the payload's tool is " "tool/tools/net6.0/any/wix.exe beneath it.\n" @@ -499,63 +593,62 @@ inline plan plan_for(options opt = {}) { "the payload is not installed for this build (mcpp provisions it " "when the feature is active on a Windows target)\n" " or set `options::tool` to name one explicitly.", - mcpp::xpkg_dir("xim", "wix")) << '\n'; - p.reason = "wix not found"; - return p; + mcpp::xpkg_dir("xim", "wix"))); } const std::string hostArch = mcpp::target_arch(); const std::string arch = wix_arch_for(hostArch); if (arch.empty()) { - std::cerr << std::format( + return refuse(p, "unknown architecture", std::format( "mcpp.dist.wix: WiX has no architecture spelling this member " "knows for '{}'. Known: x86_64 -> x64, aarch64 -> arm64.", - hostArch.empty() ? "unknown" : hostArch) << '\n'; - p.reason = "unknown architecture"; - return p; + hostArch.empty() ? "unknown" : hostArch)); } const std::string name = product_name_for(opt); - std::string wxsPath; - if (!opt.wxs.empty()) { - if (!is_file(opt.wxs)) { - std::cerr << std::format( - "mcpp.dist.wix: the definition file {} was not found", opt.wxs) << '\n'; - p.reason = "definition file not found"; - return p; - } - wxsPath = opt.wxs; - } else { + + // THE PRODUCT'S METADATA, read once for whichever definition this member + // generates: the MSI's when the project supplies none, the bundle's under + // `--format setup` when the project supplies none. + const bool generatesMsi = opt.wxs.empty(); + const bool generatesBundle = setup && opt.bundle_wxs.empty(); + std::string version, manufacturer, identity; + if (generatesMsi || generatesBundle) { const char* pv = mcpp::package_version(); const std::string rawVersion = !opt.version.empty() ? opt.version : (pv && *pv ? std::string(pv) : std::string()); if (rawVersion.empty()) { - std::cerr << "mcpp.dist.wix: no version to state. Set " - "`[package] version` or `options::version`.\n"; - p.reason = "no version"; - return p; + return refuse(p, "no version", "mcpp.dist.wix: no version to state. Set " + "`[package] version` or `options::version`."); } const auto mv = msi_version_from(rawVersion); if (!mv.ok) { - std::cerr << std::format( + return refuse(p, "version not numeric", std::format( "mcpp.dist.wix: '{}' is not a purely numeric, dot-separated " "version, which is what an MSI's Version attribute requires.", - rawVersion) << '\n'; - p.reason = "version not numeric"; - return p; + rawVersion)); } - const std::string manufacturer = manufacturer_for(opt); + version = mv.text; + manufacturer = manufacturer_for(opt); const char* nsC = mcpp::package_namespace(); const char* nmC = mcpp::package_name(); - const std::string identity = (nsC && *nsC ? std::string(nsC) : std::string()) - + "/" + (nmC && *nmC ? std::string(nmC) : std::string()); + identity = (nsC && *nsC ? std::string(nsC) : std::string()) + + "/" + (nmC && *nmC ? std::string(nmC) : std::string()); + } + + std::string wxsPath; + if (!generatesMsi) { + if (!is_file(opt.wxs)) { + return refuse(p, "definition file not found", std::format( + "mcpp.dist.wix: the definition file {} was not found", opt.wxs)); + } + wxsPath = opt.wxs; + } else { const std::string upgradeCode = !opt.upgrade_code.empty() ? opt.upgrade_code : upgrade_code_for(identity); wxsPath = (std::filesystem::path(opt.out_dir) / (name + ".wxs")).string(); - if (!write_if_different(wxsPath, wxs_document(name, manufacturer, mv.text, upgradeCode))) { - std::cerr << std::format("mcpp.dist.wix: cannot write {}", wxsPath) << '\n'; - p.reason = "cannot write definition"; - return p; + if (!write_if_different(wxsPath, wxs_document(name, manufacturer, version, upgradeCode))) { + return refuse(p, "cannot write definition", std::format("mcpp.dist.wix: cannot write {}", wxsPath)); } } @@ -581,6 +674,64 @@ inline plan plan_for(options opt = {}) { // reads, so editing the `.wxs` rebuilds the MSI and a dependency's shared // library -- which the MSI's one `File` row never names -- does not. p.inputs = { targetFile, wxsPath }; + + // `--format setup`: the bundle that chains the MSI above. + if (setup) { + const std::string extension = discover_bal_extension(opt); + if (extension.empty()) { + return refuse(p, "bootstrapper application extension not found", std::format( + "mcpp.dist.wix: a bundle with WiX's stock bootstrapper application " + "needs WixToolset.BootstrapperApplications.wixext, and none was found " + "beneath xpkg_dir(\"xim\", \"wix\") = \"{}\" at " + "bal/wixext5/WixToolset.BootstrapperApplications.wixext.dll. " + "xim:wix carries it from 5.0.2-1; or set `options::extension`.", + mcpp::xpkg_dir("xim", "wix"))); + } + std::string bundleWxs; + if (!generatesBundle) { + if (!is_file(opt.bundle_wxs)) { + return refuse(p, "bundle definition file not found", std::format( + "mcpp.dist.wix: the bundle definition file {} was not found", + opt.bundle_wxs)); + } + bundleWxs = opt.bundle_wxs; + } else { + bundleWxs = (std::filesystem::path(opt.out_dir) / (name + "-bundle.wxs")).string(); + const std::string doc = bundle_document(name, manufacturer, version, + upgrade_code_for(identity + "#bundle"), + opt.license_url); + if (!write_if_different(bundleWxs, doc)) { + return refuse(p, "cannot write bundle definition", std::format("mcpp.dist.wix: cannot write {}", bundleWxs)); + } + } + p.bundle_output = !opt.bundle_output.empty() ? opt.bundle_output + : (std::filesystem::path(opt.out_dir) + / std::format("{}-{}.exe", name, arch)).string(); + { + std::string leaf = std::filesystem::path(p.bundle_output).filename().string(); + for (std::size_t i = 0; i < leaf.size(); ++i) + if (leaf[i] >= 'A' && leaf[i] <= 'Z') leaf[i] = static_cast(leaf[i] - 'A' + 'a'); + if (leaf == "setup.exe") { + return refuse(p, "bundle named setup.exe", "mcpp.dist.wix: a bundle named setup.exe is refused by wix " + "(WIX0388: Windows loads compatibility shims into an " + "executable named like an installer). Choose another " + "`options::bundle_output`."); + } + } + p.bundle_wxs_path = bundleWxs; + p.bundle_argv = { + tool, "build", + "-arch", arch, + "-ext", extension, + "-d", "Msi=" + p.output, + "-o", p.bundle_output, + bundleWxs, + }; + // The MSI is an input, which is what orders the bundle after it and + // makes the bundle the request's one terminal artifact. + p.bundle_inputs = { p.output, bundleWxs, extension }; + } + p.applies = true; return p; } @@ -598,6 +749,17 @@ inline bool submit(const plan& p) { a.output(p.output.c_str()); a.submit(); + if (!p.bundle_argv.empty()) { + mcpp::action b; + b.id = "mcpp.dist.wix.bundle"; + b.role = "artifact"; + b.description = "BURN BUNDLE"; + for (auto const& tok : p.bundle_argv) b.arg(tok.c_str()); + for (auto const& in : p.bundle_inputs) b.input(in.c_str()); + b.output(p.bundle_output.c_str()); + b.submit(); + } + // A FLOOR ON THIS MEMBER'S OWN OUTPUT, ON THE SUCCESS PATH -- IN TWO // HALVES, BECAUSE ONE THING IS ALWAYS MEASURABLE AND THE OTHER IS NOT. // @@ -670,6 +832,7 @@ inline bool submit(const plan& p) { // format -- and makes the set unknowable for everyone else. inline bool generate(options opt = {}) { mcpp::provides_pack_format("msi"); + mcpp::provides_pack_format("setup"); return submit(plan_for(std::move(opt))); } diff --git a/mcpp.toml b/mcpp.toml index 4963b1e..45e15c1 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -388,8 +388,15 @@ implies = ["surface"] # The per-host table stays the right shape only at the top level, where # platform keys select the HOST rather than restating a predicate the # section has already applied. +# +# 5.0.2-1, NOT 5.0.2, BECAUSE `--format setup` NEEDS THE EXTENSION. The recipe +# revision adds `WixToolset.BootstrapperApplications.wixext` beside the tool, +# and a bundle with WiX's stock bootstrapper application is refused without it +# (WIX0200). An installation made under 5.0.2 keeps the payloads it fetched, +# because xlings does not re-run the install hook of an installed version, so +# the pin names the revision rather than the release. [target.windows.feature-xlings.dist-wix] -"xim:wix" = "5.0.2" +"xim:wix" = "5.0.2-1" # `dist-apple` declares no payload for `codesign`, `ditto` or `hdiutil`: they are # part of macOS and Xcode, which are not redistributable, so the member runs the diff --git a/tests/msi-consumer/check-setup.sh b/tests/msi-consumer/check-setup.sh new file mode 100755 index 0000000..5197529 --- /dev/null +++ b/tests/msi-consumer/check-setup.sh @@ -0,0 +1,54 @@ +#!/usr/bin/env bash +# dist-wix `--format setup` on Windows (mcpp#634, B4). +# +# 1. `mcpp pack --format setup` writes a Burn bundle named after the product +# and the architecture, never `setup.exe`, beside the MSI it chains. +# 2. The bundle carries that MSI: `wix burn extract` takes the attached +# container apart, and the package inside is byte-for-byte the MSI the +# first action wrote. +# +# The bundle's user interface is WiX's stock bootstrapper application, which +# `wix build` finds only through the extension `xim:wix` 5.0.2-1 carries; a +# bundle definition that names it builds only with that extension loaded. +# +# Usage: MCPP= ./check-setup.sh (run from this directory, on Windows, under Git Bash) +set -eu + +MCPP="${MCPP:-mcpp}" +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +reading() { printf 'READING %s: %s\n' "$1" "$2"; } +packed() { sed -n 's/^ *Packed //p' "$1" | tail -1 | tr -d '\r'; } +size() { stat -c %s "$1" 2>/dev/null || stat -f %z "$1"; } + +# ── 1. the bundle ────────────────────────────────────────────────────────── +echo "== 1. mcpp pack --format setup ==" +"$MCPP" pack --format setup > setup-pack.log 2>&1 || fail "mcpp pack --format setup failed" setup-pack.log +bundle=$(packed setup-pack.log) +reading packed "$bundle" +case "$bundle" in *MsiConsumer-x64.exe) ;; *) fail "the pack reported '$bundle', not MsiConsumer-x64.exe" setup-pack.log ;; esac +[ -f "$bundle" ] || fail "the reported bundle $bundle does not exist" setup-pack.log +msi="$(dirname "$bundle")/MsiConsumer-x64.msi" +[ -f "$msi" ] || fail "no MSI beside the bundle at $msi" setup-pack.log +if find target -iname 'setup.exe' | grep -q .; then fail "a file named setup.exe was written"; fi +reading sizes "bundle $(size "$bundle") bytes, msi $(size "$msi") bytes" +echo "ok: $(basename "$bundle") beside $(basename "$msi")" + +# ── 2. the MSI inside the bundle ─────────────────────────────────────────── +echo "== 2. the bundle carries the MSI ==" +"$MCPP" self env > setup-env.txt +home=$(awk -F'= *' '/^MCPP_HOME/{print $2; exit}' setup-env.txt | tr -d '\r') +[ -n "$home" ] || fail "could not read MCPP_HOME" setup-env.txt +command -v cygpath > /dev/null && home=$(cygpath -u "$home") +wix="$home/registry/data/xpkgs/xim-x-wix/5.0.2-1/tool/tools/net6.0/any/wix.exe" +[ -f "$wix" ] || fail "no wix.exe of xim:wix 5.0.2-1 at $wix" +out="$PWD/setup-extract" +rm -rf "$out" +"$wix" burn extract "$(cygpath -w "$bundle")" -o "$(cygpath -w "$out")" > setup-extract.log 2>&1 \ + || fail "wix burn extract refused the bundle" setup-extract.log +inside=$(find "$out" -type f -iname '*.msi') +reading extracted "$(find "$out" -type f | sed "s|^$out/||" | tr '\n' ' ')" +[ "$(printf '%s\n' "$inside" | grep -c .)" = 1 ] || fail "expected one MSI in the bundle, found: $(echo $inside)" setup-extract.log +cmp -s "$inside" "$msi" || fail "the MSI inside the bundle ($(size "$inside") bytes) differs from $msi ($(size "$msi") bytes)" +echo "ok: the bundle's container holds the MSI, byte-for-byte" + +echo "PASS: dist-wix writes a Burn bundle that chains the MSI" From 4c99c66759934e1707a29bcd93a357cb475ef487 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Mon, 14 Sep 2026 16:03:19 +0800 Subject: [PATCH 05/13] feat(rules-metal): Metal shaders to Metal libraries through xcrun metal and metallib (mcpp#634) The rule in rules-spirv's shape: `.metal` is a device extension the feature declares, one `xcrun --sdk metal -MMD -MF` action per compilation with the compiler's dependency file as the action's depfile, and one `metallib` action per library, deployed beside the program under metallib/. compile(shaders) compiles one source several times with its own definitions; options::library links every shader into one library. The toolchain is located, not installed: the rule asks `xcrun --sdk --show-sdk-path`, then `--find metal` and `--find metallib`, and refuses naming the command that answered nothing. A shader on a row other than macOS or iOS is refused naming the row. tests/metal-consumer: two shaders and a variant of one; the program checks each library's magic. check-metal.sh adds the depfile criterion (a header edit recompiles only the shaders that include it) and records an absent toolchain as unmeasured. tests/all-rules-compile names the feature, so the module compiles on every host (Linux: ok). --- mcpp.toml | 10 + rules/metal.cppm | 295 +++++++++++++++++++++ tests/all-rules-compile/build.mcpp | 5 + tests/all-rules-compile/mcpp.toml | 4 +- tests/metal-consumer/build.mcpp | 18 ++ tests/metal-consumer/check-metal.sh | 82 ++++++ tests/metal-consumer/mcpp.toml | 34 +++ tests/metal-consumer/shaders/scale.metal | 7 + tests/metal-consumer/shaders/tint.metal | 8 + tests/metal-consumer/shaders/tint_common.h | 7 + tests/metal-consumer/src/main.cpp | 29 ++ 11 files changed, 497 insertions(+), 2 deletions(-) create mode 100644 rules/metal.cppm create mode 100644 tests/metal-consumer/build.mcpp create mode 100755 tests/metal-consumer/check-metal.sh create mode 100644 tests/metal-consumer/mcpp.toml create mode 100644 tests/metal-consumer/shaders/scale.metal create mode 100644 tests/metal-consumer/shaders/tint.metal create mode 100644 tests/metal-consumer/shaders/tint_common.h create mode 100644 tests/metal-consumer/src/main.cpp diff --git a/mcpp.toml b/mcpp.toml index 45e15c1..f5c156f 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -84,6 +84,16 @@ implies = ["surface"] rule_module = "mcpp.rules.cuda" device_extensions = [".cu"] +# `.metal` shaders become Metal libraries through `xcrun metal` and `xcrun +# metallib`, the toolchain Xcode carries. No payload is declared, for the reason +# `dist-apple` gives for `codesign`: Xcode is not redistributable, so the rule +# locates the toolchain and refuses naming the command that found nothing. +[features.rules-metal] +sources = ["rules/metal.cppm"] +implies = ["surface"] +rule_module = "mcpp.rules.metal" +device_extensions = [".metal"] + [features.rules-hip] sources = ["rules/hip.cppm"] implies = ["surface"] diff --git a/rules/metal.cppm b/rules/metal.cppm new file mode 100644 index 0000000..9c8b219 --- /dev/null +++ b/rules/metal.cppm @@ -0,0 +1,295 @@ +// mcpp.rules.metal -- how a Metal shader becomes a Metal library, stated once. +// +// THE DIVISION OF LABOUR IS `mcpp.rules.spirv`'S. The ENGINE owns the graph: +// `.metal` is a device extension this feature declares, so a `.metal` file the +// project names in `[build] sources` reaches this rule through +// `mcpp::device_sources()` instead of being refused, and every command below is +// an action with declared inputs and outputs. The RULE owns the spelling: which +// SDK, which two tools, which flags, and where the library is placed. +// +// TWO TOOLS AND ONE INTERMEDIATE. `metal` compiles a shader to Apple's +// intermediate representation (`.air`), and `metallib` links one or more of +// those into a Metal library (`.metallib`), the file an application loads with +// `newLibraryWithURL:` or, named `default.metallib` in its bundle's resources, +// with `newDefaultLibrary`. Each is one action, so an edited shader recompiles +// its own `.air` and relinks only the libraries that contain it. +// +// THE TOOLCHAIN IS LOCATED, NOT INSTALLED. Both tools ship with Xcode, which is +// not redistributable, so no payload is declared. Before planning anything the +// rule asks `xcrun --sdk --show-sdk-path`, then `xcrun --sdk --find +// metal` and `--find metallib`, and refuses naming the command that answered +// nothing: a missing SDK and a missing compiler are different remedies. Xcode +// 26 installs the Metal toolchain as a separate component (`xcodebuild +// -downloadComponent MetalToolchain`), which is the absence a machine with +// Xcode and its SDKs can still have. +// The commands themselves run through `xcrun --sdk `, because `xcrun` +// sets the SDK the compiler targets; an absolute tool path would compile for +// macOS whatever the row is. +// +// THE LIBRARY IS A FILE BESIDE THE PROGRAM. It is placed with `mcpp::deploy` +// under `options::deploy_to` (`metallib/` by default), relative to the +// program's directory, which is where `dist-apple` finds a deployed file and +// maps it under the bundle's resource directory. +// +// THE SAME SOURCE MAY PRODUCE SEVERAL LIBRARIES. A renderer that compiles one +// fragment shader once per blend mode passes one `shader` per output, each with +// its own definitions and name; the sources need not be in the project's tree. + +module; +#include + +export module mcpp.rules.metal; + +import std; +import mcpp; +import mcpp.plugins; + +// Nothing here uses `std::println`: both of its overloads reach into the libc++ +// dylib for symbols macOS 14 does not ship, so a build program that printed +// with it compiled and then failed to link. The measurement is in +// `rules/spirv.cppm`. + +export namespace mcpp::rules::metal { + +struct options { + // `-D` for the Metal preprocessor and `-I` for `#include`, applied to every + // shader. Relative include directories resolve against the package root. + std::vector defines; + std::vector includes; + // `-std=` (`metal3.1`, `ios-metal2.4`). Empty lets the compiler take + // the default of the SDK it compiles against. + std::string language_standard; + // Where the libraries are placed, relative to the program's directory: + // `mcpp::deploy`'s destination. + std::string deploy_to = "metallib"; + // Empty compiles one library per shader, named after the shader. A name + // links every shader into one library `.metallib`; `default` gives + // the library `newDefaultLibrary` finds in a bundle's resources. + std::string library; + // The SDK `xcrun` compiles against. Empty derives it from the target row: + // `macosx`, `iphoneos` or `iphonesimulator`. + std::string sdk; + std::string out_dir = std::string(mcpp::out_dir()); +}; + +// One compilation of a shader. +struct shader { + // Absolute, or relative to the package root. + std::string source; + // The stem of the library it produces. Empty means the source's stem. + std::string name; + // Definitions for this compilation alone, after `options::defines`. + std::vector defines; +}; + +// ─── What the engine said ────────────────────────────────────────────────── + +// The SDK a target row compiles against, or empty on a row Metal does not +// serve. +inline std::string sdk_for(const options& opt) { + if (!opt.sdk.empty()) return opt.sdk; + const std::string os = mcpp::target_os(); + if (os == "macos") return "macosx"; + if (os == "ios") + return std::string(mcpp::target_env()) == "sim" ? "iphonesimulator" : "iphoneos"; + return {}; +} + +// `mcpp::device_sources()` is the package's whole device set, one path per +// line, and a rule takes the extensions it claims (see `rules/spirv.cppm`). +inline std::vector device_shaders() { + std::vector out; + const std::string all = mcpp::device_sources(); + std::size_t i = 0; + while (i <= all.size()) { + auto nl = all.find('\n', i); + std::string one = all.substr(i, nl == std::string::npos ? std::string::npos : nl - i); + i = nl == std::string::npos ? all.size() + 1 : nl + 1; + while (!one.empty() && (one.back() == ' ' || one.back() == '\r')) one.pop_back(); + if (one.size() > 6 && one.substr(one.size() - 6) == ".metal") out.push_back(one); + } + return out; +} + +// ─── The toolchain ───────────────────────────────────────────────────────── + +// The first line a command prints, or empty. `popen` is POSIX; this module is +// compiled on every host (`tests/all-rules-compile`), so Windows is spelled. +inline std::string first_line_of(const std::string& cmd) { +#if defined(_WIN32) + FILE* p = ::_popen(cmd.c_str(), "r"); +#else + FILE* p = ::popen(cmd.c_str(), "r"); +#endif + if (!p) return {}; + char buf[1024]; + std::string line; + if (std::fgets(buf, sizeof buf, p)) line = buf; +#if defined(_WIN32) + ::_pclose(p); +#else + ::pclose(p); +#endif + while (!line.empty() && (line.back() == '\n' || line.back() == '\r')) line.pop_back(); + return line; +} + +// The SDK's path, or empty when `xcrun` cannot locate it. +inline std::string sdk_path(const std::string& sdk) { +#if defined(_WIN32) + (void)sdk; + return {}; +#else + const std::string path = first_line_of( + "/usr/bin/xcrun --sdk " + sdk + " --show-sdk-path 2>/dev/null"); + std::error_code ec; + return !path.empty() && std::filesystem::is_directory(path, ec) ? path : std::string(); +#endif +} + +inline std::string find_tool(const std::string& sdk, const char* tool) { +#if defined(_WIN32) + (void)sdk; (void)tool; + return {}; +#else + const std::string path = first_line_of( + "/usr/bin/xcrun --sdk " + sdk + " --find " + tool + " 2>/dev/null"); + std::error_code ec; + return !path.empty() && std::filesystem::is_regular_file(path, ec) ? path : std::string(); +#endif +} + +inline std::string stem_of(const std::string& path) { + return std::filesystem::path(path).stem().string(); +} + +// ─── The rule ────────────────────────────────────────────────────────────── + +inline bool compile(std::span shaders, options opt = {}) { + if (shaders.empty()) return true; + + const std::string sdk = sdk_for(opt); + if (sdk.empty()) { + std::cerr << std::format( + "mcpp.rules.metal: {} Metal shader(s) reached this rule on a '{}' " + "target; Metal compiles for macOS and iOS only. Condition the " + "shaders on the row, e.g. [target.'cfg(any(os = \"macos\", os = \"ios\"))'.build] " + "sources = [\"shaders/*.metal\"].", + shaders.size(), std::string(mcpp::target_os())) << '\n'; + return false; + } + if (sdk_path(sdk).empty()) { + std::cerr << std::format( + "mcpp.rules.metal: the {0} SDK was not found: `xcrun --sdk {0} " + "--show-sdk-path` answered nothing. The macOS SDK ships with Xcode " + "and with the Command Line Tools, the iOS SDKs with Xcode alone.", + sdk) << '\n'; + return false; + } + const std::string metalTool = find_tool(sdk, "metal"); + const std::string metallibTool = find_tool(sdk, "metallib"); + if (metalTool.empty() || metallibTool.empty()) { + std::cerr << std::format( + "mcpp.rules.metal: the Metal toolchain was not found for the {0} SDK: " + "`xcrun --sdk {0} --find {1}` answered nothing. It ships with Xcode, " + "and Xcode 26 installs it as a separate component: " + "`xcodebuild -downloadComponent MetalToolchain`.", + sdk, metalTool.empty() ? "metal" : "metallib") << '\n'; + return false; + } + + const std::string root = mcpp::manifest_dir(); + const auto gen = std::filesystem::path(opt.out_dir) / "metal"; + std::error_code ec; + std::filesystem::create_directories(gen, ec); + + // Two compilations with one output name would be two actions writing one + // file; refused by naming both, as `rules/spirv.cppm` refuses two shaders + // mapping to one header. + { + std::map seen; + for (auto const& s : shaders) { + const std::string name = s.name.empty() ? stem_of(s.source) : s.name; + auto [it, fresh] = seen.try_emplace(name, s.source); + if (!fresh) { + std::cerr << std::format( + "mcpp.rules.metal: two compilations produce `{}.air`: {} and {}. " + "Give one of them a `shader::name`.", name, it->second, s.source) << '\n'; + return false; + } + } + } + + std::vector airs; + for (auto const& s : shaders) { + const std::string source = std::filesystem::path(s.source).is_absolute() + ? s.source : root + "/" + s.source; + const std::string name = s.name.empty() ? stem_of(s.source) : s.name; + const std::string air = (gen / (name + ".air")).string(); + const std::string dep = air + ".d"; + + const std::string id = "metal:" + name; + const std::string desc = "METAL " + name; + mcpp::action a; + a.id = id.c_str(); + a.role = "source"; + a.description = desc.c_str(); + a.arg("/usr/bin/xcrun").arg("--sdk").arg(sdk.c_str()).arg("metal"); + if (!opt.language_standard.empty()) + a.arg(("-std=" + opt.language_standard).c_str()); + for (auto const& d : opt.defines) a.arg(("-D" + d).c_str()); + for (auto const& d : s.defines) a.arg(("-D" + d).c_str()); + for (auto const& i : opt.includes) + a.arg(("-I" + (std::filesystem::path(i).is_absolute() ? i : root + "/" + i)).c_str()); + // WHAT THE SHADER `#include`s, which only the compiler knows. The + // `metal` driver is clang's and writes a Makefile-style dependency + // file as clang does. + a.arg("-MMD").arg("-MF").arg(dep.c_str()); + a.depfile = dep.c_str(); + a.arg("-c").arg(source.c_str()).arg("-o").arg(air.c_str()); + a.input(source.c_str()); + a.output(air.c_str()); + a.submit(); + airs.push_back(air); + } + + auto link = [&](const std::string& name, std::span inputs) { + const std::string lib = (gen / (name + ".metallib")).string(); + const std::string id = "metallib:" + name; + const std::string desc = "METALLIB " + name; + mcpp::action a; + a.id = id.c_str(); + a.role = "source"; + a.description = desc.c_str(); + a.arg("/usr/bin/xcrun").arg("--sdk").arg(sdk.c_str()).arg("metallib"); + for (auto const& in : inputs) { a.arg(in.c_str()); a.input(in.c_str()); } + a.arg("-o").arg(lib.c_str()); + a.output(lib.c_str()); + a.submit(); + mcpp::deploy(lib.c_str(), opt.deploy_to.c_str()); + }; + if (opt.library.empty()) { + for (std::size_t i = 0; i < shaders.size(); ++i) { + const std::string name = shaders[i].name.empty() ? stem_of(shaders[i].source) + : shaders[i].name; + link(name, std::span(&airs[i], 1)); + } + } else { + link(opt.library, airs); + } + + mcpp::fact("mcpp.plugins", std::string(mcpp::plugins::version).c_str()); + return true; +} + +// The shaders the project named in `[build] sources`, one library each unless +// `options::library` names one for all of them. A build whose sources name no +// `.metal` file has nothing to do, which is not a mistake: a project names the +// shaders on the rows that compile them. +inline bool compile(options opt = {}) { + std::vector list; + for (auto const& path : device_shaders()) list.push_back({ path, {}, {} }); + return compile(std::span(list), std::move(opt)); +} + +} // namespace mcpp::rules::metal diff --git a/tests/all-rules-compile/build.mcpp b/tests/all-rules-compile/build.mcpp index 55de2fe..a607850 100644 --- a/tests/all-rules-compile/build.mcpp +++ b/tests/all-rules-compile/build.mcpp @@ -8,6 +8,7 @@ import mcpp; import mcpp.rules.ascendc; import mcpp.rules.cuda; import mcpp.rules.hip; +import mcpp.rules.metal; import mcpp.rules.slang; import mcpp.rules.spirv; import mcpp.rules.sycl; @@ -26,6 +27,10 @@ int main() { bool ok = mcpp::rules::ascendc::compile() && mcpp::rules::cuda::compile() && mcpp::rules::hip::compile() + // No `.metal` source is named, so the Metal rule plans nothing and + // never asks `xcrun`, on the hosts that have it and the ones that + // do not. + && mcpp::rules::metal::compile() && mcpp::rules::slang::compile() && mcpp::rules::spirv::compile() && mcpp::rules::sycl::compile(); diff --git a/tests/all-rules-compile/mcpp.toml b/tests/all-rules-compile/mcpp.toml index 738eceb..aa1a27c 100644 --- a/tests/all-rules-compile/mcpp.toml +++ b/tests/all-rules-compile/mcpp.toml @@ -40,8 +40,8 @@ import_std = true # list is one whose host-dependent code is compiled on one platform only. [build-dependencies.mcpp] plugins = { path = "../..", features = [ - "rules-ascendc", "rules-cuda", "rules-hip", "rules-slang", "rules-spirv", - "rules-sycl", + "rules-ascendc", "rules-cuda", "rules-hip", "rules-metal", "rules-slang", + "rules-spirv", "rules-sycl", "tools-embed", "tools-island", "dist-appimage", "dist-wix", "dist-apple", "dist-web", "dist-apk", ], host-module = true } diff --git a/tests/metal-consumer/build.mcpp b/tests/metal-consumer/build.mcpp new file mode 100644 index 0000000..29e15e4 --- /dev/null +++ b/tests/metal-consumer/build.mcpp @@ -0,0 +1,18 @@ +import std; +import mcpp; +import mcpp.rules.metal; + +int main() { + // The shaders `[build] sources` names: one library each. + if (!mcpp::rules::metal::compile()) return 1; + + // A variant of one of them, compiled again with a definition of its own + // and named for it -- one `shader` per output. + if (std::string(mcpp::target_os()) == "macos") { + const std::vector variants = { + { "shaders/tint.metal", "tint_red", { "TINT_RED=1" } }, + }; + if (!mcpp::rules::metal::compile(variants)) return 1; + } + return 0; +} diff --git a/tests/metal-consumer/check-metal.sh b/tests/metal-consumer/check-metal.sh new file mode 100755 index 0000000..eedf37c --- /dev/null +++ b/tests/metal-consumer/check-metal.sh @@ -0,0 +1,82 @@ +#!/usr/bin/env bash +# rules-metal on macOS (mcpp#634, B6). +# +# 1. The fixture's two shaders, and a variant of one of them, compile to three +# Metal libraries placed beside the program; the program opens each one and +# finds the Metal library magic. +# 2. The dependency file the compiler writes reaches the graph: editing the +# header two of the compilations include recompiles those two and not the +# third, which no rule could have declared as an input. +# +# THE TOOLCHAIN IS A PROPERTY OF THE RUNNER. When `xcrun --sdk macosx --find +# metal` or `--find metallib` answers nothing, the rule refuses naming that +# command. This script then asserts the refusal, records that the two criteria +# above are unmeasured on the runner, and exits 0 with a warning annotation +# saying so, rather than reporting a library it did not build. +# +# Usage: MCPP= ./check-metal.sh (run from this directory, on macOS) +set -eu + +MCPP="${MCPP:-mcpp}" +fail() { echo "FAIL: $1"; shift; for f in "$@"; do echo "--- $f ---"; cat "$f" 2>/dev/null; done; exit 1; } +reading() { printf 'READING %s: %s\n' "$1" "$2"; } + +missing="" +for tool in metal metallib; do + found=$(xcrun --sdk macosx --find "$tool" 2>/dev/null || true) + reading "xcrun-find-$tool" "${found:-nothing}" + [ -n "$found" ] || { [ -n "$missing" ] || missing="$tool"; } +done + +rm -rf target +if [ -n "$missing" ]; then + rc=0 + "$MCPP" build > metal-build.log 2>&1 || rc=$? + [ "$rc" -ne 0 ] || fail "the build succeeded, and xcrun finds no $missing" metal-build.log + grep -q "xcrun --sdk macosx --find $missing" metal-build.log \ + || fail "the build failed without naming the command that found no $missing" metal-build.log + reading refusal "$(grep -m1 'mcpp.rules.metal:' metal-build.log)" + reading criterion "UNMEASURED on this runner: no $missing, so no Metal library was built" + echo "::warning title=rules-metal unmeasured::xcrun --sdk macosx --find $missing answered nothing on this runner; the rule refused naming that command, and no Metal library was built or checked" + exit 0 +fi +reading metal-version "$(xcrun --sdk macosx metal --version 2>&1 | head -1)" + +# ── 1. three libraries beside the program ───────────────────────────────── +echo "== 1. the shaders compile to Metal libraries the program finds ==" +"$MCPP" build > metal-build.log 2>&1 || fail "mcpp build failed" metal-build.log +gen=target/.build-mcpp/out/metal +for name in scale tint tint_red; do + [ -f "$gen/$name.air" ] || fail "no $gen/$name.air" metal-build.log + [ -f "$gen/$name.metallib" ] || fail "no $gen/$name.metallib" metal-build.log + magic=$(head -c 4 "$gen/$name.metallib") + reading "$name.metallib" "$(wc -c < "$gen/$name.metallib" | tr -d ' ') bytes, magic '$magic'" + [ "$magic" = MTLB ] || fail "$gen/$name.metallib does not begin with MTLB" +done +rc=0 +"$MCPP" run > metal-run.log 2>&1 || rc=$? +[ "$rc" -eq 0 ] || fail "the program exited $rc" metal-run.log +grep -qx 'metal-consumer ok' metal-run.log || fail "the program did not find its libraries" metal-run.log +echo "ok: scale, tint and tint_red are Metal libraries, deployed where the program looks" + +# ── 2. the dependency file ───────────────────────────────────────────────── +echo "== 2. an edited header recompiles the shaders that include it ==" +header=shaders/tint_common.h +cp "$header" "$header.orig" +trap 'mv -f "$header.orig" "$header"' EXIT +stamp() { stat -f %m "$gen/$1.air"; } +before="$(stamp scale) $(stamp tint) $(stamp tint_red)" +sleep 2 +sed 's/#define TINT_R 0.0h/#define TINT_R 0.5h/' "$header.orig" > "$header" +cmp -s "$header" "$header.orig" && fail "the header edit changed nothing" "$header" +"$MCPP" build > metal-rebuild.log 2>&1 || fail "the rebuild failed" metal-rebuild.log +after="$(stamp scale) $(stamp tint) $(stamp tint_red)" +reading air-mtimes "scale tint tint_red: before $before, after $after" +set -- $before; b_scale=$1 b_tint=$2 b_red=$3 +set -- $after; a_scale=$1 a_tint=$2 a_red=$3 +[ "$a_tint" != "$b_tint" ] || fail "tint.air was not recompiled after its header changed" metal-rebuild.log +[ "$a_red" != "$b_red" ] || fail "tint_red.air was not recompiled after its header changed" metal-rebuild.log +[ "$a_scale" = "$b_scale" ] || fail "scale.air was recompiled, and it includes nothing that changed" metal-rebuild.log +echo "ok: the two compilations that include the header were recompiled, the third was not" + +echo "PASS: rules-metal compiles Metal libraries, and the compiler's dependency file reaches the graph" diff --git a/tests/metal-consumer/mcpp.toml b/tests/metal-consumer/mcpp.toml new file mode 100644 index 0000000..1bf3f70 --- /dev/null +++ b/tests/metal-consumer/mcpp.toml @@ -0,0 +1,34 @@ +# Fixture: rules-metal (mcpp#634, B6). +# +# Two libraries from the shaders the project names, one per shader, and a +# variant compiled from one of those sources with a definition of its own, the +# shape a renderer takes when it compiles one fragment shader per blend mode. +# The program opens each library it expects beside itself and checks the Metal +# library magic, so no GPU is involved. +# +# The shaders are sources on the macOS row only: a `.metal` file is a device +# source, and a row that names none plans nothing. +[package] +name = "metal-consumer" +version = "0.1.0" +description = "Fixture: Metal shaders compiled to Metal libraries by rules-metal" +license = "Apache-2.0" +authors = ["mcpp-community"] + +[language] +standard = "c++23" +modules = true +import_std = true + +[build-dependencies.mcpp] +plugins = { path = "../..", features = ["rules-metal"], host-module = true } + +[build] +sources = ["src/*.cpp"] + +[target.'cfg(os = "macos")'.build] +sources = ["shaders/*.metal"] + +[targets.metal-consumer] +kind = "bin" +main = "src/main.cpp" diff --git a/tests/metal-consumer/shaders/scale.metal b/tests/metal-consumer/shaders/scale.metal new file mode 100644 index 0000000..2c0529f --- /dev/null +++ b/tests/metal-consumer/shaders/scale.metal @@ -0,0 +1,7 @@ +#include +using namespace metal; + +kernel void scale_values(buffer values [[buffer(0)]], + uint index [[thread_position_in_grid]]) { + values[index] = values[index] * 2.0; +} diff --git a/tests/metal-consumer/shaders/tint.metal b/tests/metal-consumer/shaders/tint.metal new file mode 100644 index 0000000..9ce9010 --- /dev/null +++ b/tests/metal-consumer/shaders/tint.metal @@ -0,0 +1,8 @@ +#include +using namespace metal; + +#include "tint_common.h" + +fragment half4 tint_fragment(float4 position [[position]]) { + return half4(TINT_R, 0.25h, 0.5h, 1.0h); +} diff --git a/tests/metal-consumer/shaders/tint_common.h b/tests/metal-consumer/shaders/tint_common.h new file mode 100644 index 0000000..1a6374a --- /dev/null +++ b/tests/metal-consumer/shaders/tint_common.h @@ -0,0 +1,7 @@ +#pragma once + +#if defined(TINT_RED) +#define TINT_R 1.0h +#else +#define TINT_R 0.0h +#endif diff --git a/tests/metal-consumer/src/main.cpp b/tests/metal-consumer/src/main.cpp new file mode 100644 index 0000000..276df89 --- /dev/null +++ b/tests/metal-consumer/src/main.cpp @@ -0,0 +1,29 @@ +#include +#include +#include + +// Opens the libraries the build placed beside the program and checks the +// Metal library magic ("MTLB"), so the criterion is the file and no GPU. +int main(int, char** argv) { +#if defined(__APPLE__) + std::string dir = argv[0]; + const auto slash = dir.rfind('/'); + dir = slash == std::string::npos ? std::string(".") : dir.substr(0, slash); + for (const char* name : { "scale", "tint", "tint_red" }) { + const std::string path = dir + "/metallib/" + name + ".metallib"; + FILE* f = std::fopen(path.c_str(), "rb"); + if (!f) { std::printf("missing %s\n", path.c_str()); return 1; } + char magic[4] = {0}; + const auto n = std::fread(magic, 1, 4, f); + std::fclose(f); + if (n != 4 || std::memcmp(magic, "MTLB", 4) != 0) { + std::printf("not a Metal library: %s\n", path.c_str()); + return 1; + } + } +#else + (void)argv; +#endif + std::printf("metal-consumer ok\n"); + return 0; +} From 9a87f5c96c8cdca0e33d6c45b8bd3ec7c3624fe4 Mon Sep 17 00:00:00 2001 From: sunrisepeak Date: Mon, 14 Sep 2026 16:04:58 +0800 Subject: [PATCH 06/13] 0.10.0: CI measures P1 to P7 against the engine under review, the iOS fixture exits 7, and the README states the new formats, member and floor (mcpp#634) CI. `.github/scripts/mcpp-under-review.sh` builds mcpp-community/mcpp at MCPP_SOURCE_REF with the released MCPP_VERSION (2026.9.14.1) and points $MCPP at the result, in both jobs that fetch mcpp; the step after it fails unless the steps run that binary and it prints the version the build printed. A dispatch input names the reference; the workflow-level default is feat/634-cmake-parity for development and is removed before merge, when MCPP_VERSION names the release that carries that engine. New steps: dist-apk's closure criteria and dist-apple's plan-level checks on Linux; on macos-15 the framework bundle (load command, codesign --verify --deep --strict, the program with and without its framework, mcpp run --format app, hdiutil) and rules-metal; on windows-2022 the Burn bundle. Each has a timeout; no step uploads. The iOS fixture exits 7 and the step asserts 7 through simctl-run 0.3.0 (xim:apple-simulator-tools, pinned), with an `od -c` diagnostic in place of `cat -A`, which BSD cat refuses. No Android fixture pins android-platform-tools. The version is 0.10.0 in mcpp.toml and src/plugins.cppm. --- .github/scripts/mcpp-under-review.sh | 66 ++++++ .github/workflows/ci.yml | 190 +++++++++++++++--- README.md | 17 +- mcpp.toml | 2 +- src/plugins.cppm | 2 +- .../apk-consumer-shared/check-apk-closure.sh | 6 +- .../check-apple-bundle.sh | 20 +- tests/ios-app-consumer/mcpp.toml | 12 +- tests/ios-app-consumer/src/main.cpp | 6 +- tests/msi-consumer/check-setup.sh | 12 +- 10 files changed, 272 insertions(+), 61 deletions(-) create mode 100755 .github/scripts/mcpp-under-review.sh diff --git a/.github/scripts/mcpp-under-review.sh b/.github/scripts/mcpp-under-review.sh new file mode 100755 index 0000000..e561deb --- /dev/null +++ b/.github/scripts/mcpp-under-review.sh @@ -0,0 +1,66 @@ +#!/usr/bin/env bash +# The mcpp a job's steps run, when that is not the released one. +# +# `MCPP_SOURCE_REF` names a branch or tag of mcpp-community/mcpp. Empty, the +# steps run the release `MCPP_VERSION` names, which the step before this one +# fetched into `$MCPP`. Set, this script builds mcpp at that reference with the +# released mcpp and points `$MCPP` at the result, so that a change to the engine +# is measured against this collection before either is released. +# +# A SCRIPT, NOT A STEP COPIED INTO EACH JOB. Two copies drift, and a job that +# installs mcpp without this channel builds manifests written for the engine +# under review with the released engine, which accepts a key it does not know +# and proceeds without the semantics the key asks for. +# +# WHAT THIS SCRIPT WRITES IS NOT EVIDENCE THAT IT TOOK EFFECT. `GITHUB_ENV` +# governs the steps that follow, so the job's next step compares what `$MCPP` +# names and prints with what this script built. +# +# Environment: MCPP (the released mcpp), MCPP_SOURCE_REF, RUNNER_TEMP, GITHUB_ENV. +set -euo pipefail + +: "${MCPP:?the released mcpp, which the step before this one fetches}" +: "${GITHUB_ENV:?}" +: "${RUNNER_TEMP:?}" + +echo "MCPP_RELEASED=$MCPP" >> "$GITHUB_ENV" +if [ -z "${MCPP_SOURCE_REF:-}" ]; then + echo "MCPP_SOURCE_REF is empty: the steps run the released $("$MCPP" --version | head -1)" + exit 0 +fi + +# In Git Bash on Windows `RUNNER_TEMP` is a Windows path (`D:\a\_temp`), and the +# binary path this script exports is used by bash in every later step, so the +# directory is spelled in bash's own syntax. +temp="$RUNNER_TEMP" +if command -v cygpath > /dev/null; then temp=$(cygpath -u "$temp"); fi +src="$temp/mcpp-src" +rm -rf "$src" +git clone --quiet --depth 1 --branch "$MCPP_SOURCE_REF" \ + https://github.com/mcpp-community/mcpp.git "$src" +echo "READING source: mcpp-community/mcpp $MCPP_SOURCE_REF at $(git -C "$src" rev-parse HEAD)" + +# The clone's `.xlings.json` pins the mcpp that builds mcpp in that repository's +# own CI, and the pin does not move with this job's release: a build inside the +# checkout obeys it and installs a version this job did not choose. Removed, the +# released mcpp above builds the source. +rm -f "$src/.xlings.json" + +(cd "$src" && "$MCPP" build) + +# A fresh clone holds no earlier build, so what remains is this build's product; +# the count is asserted rather than assumed. +built=$(find "$src/target" -type f \( -name mcpp -o -name mcpp.exe \)) +count=$(printf '%s\n' "$built" | grep -c . || true) +if [ "$count" != 1 ]; then + echo "::error::expected one mcpp binary from $MCPP_SOURCE_REF, found $count" + printf '%s\n' "$built" | sed 's/^/ /' + exit 1 +fi +version=$("$built" --version | head -1) +echo "READING under review: $version at $built" +{ + echo "MCPP=$built" + echo "MCPP_UNDER_REVIEW=$built" + echo "MCPP_UNDER_REVIEW_VERSION=$version" +} >> "$GITHUB_ENV" diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index f75f4f7..f703de6 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -5,6 +5,11 @@ on: branches: [main] pull_request: workflow_dispatch: + inputs: + mcpp_source_ref: + description: "A branch or tag of mcpp-community/mcpp to build and run in place of the release MCPP_VERSION names" + required: false + default: "" env: # The mcpp release the consumers build with. Raising it is what admits a @@ -29,7 +34,24 @@ env: # every dispatched format was unreachable on macOS, including one that never # reads the staged tree. Staging is a service to the provider in 2026.9.11.2, # and `dist-apple` has been unreachable, not broken, since it was written. - MCPP_VERSION: 2026.9.13.2 + # + # 2026.9.14.1 WHILE mcpp#634 IS DEVELOPED, AS THE RELEASE THAT BUILDS THE + # ENGINE UNDER REVIEW. 0.10.0's `dist-apk` and `dist-apple` read the closure + # the engine stages, which needs 2026.9.14.2; this pin names that release + # before 0.10.0 merges. + MCPP_VERSION: 2026.9.14.1 + # THE ENGINE UNDER REVIEW, BUILT FROM SOURCE WHEN THIS IS NOT EMPTY. + # + # `.github/scripts/mcpp-under-review.sh` builds mcpp-community/mcpp at this + # reference with the release above and points `$MCPP` at the result, in each + # job that fetches mcpp, and the step after it checks that the steps run what + # it built. A dispatch names the reference through its input. The default is + # the development branch of mcpp#634, whose engine stages the closures + # `dist-apk` and `dist-apple` read, runs a distributable through the runner + # named after its format, and leaves an `@executable_path` rpath as written; + # it is removed before 0.10.0 merges, when MCPP_VERSION names the release that + # carries that engine. + MCPP_SOURCE_REF: ${{ inputs.mcpp_source_ref || 'feat/634-cmake-parity' }} # PINNED, AND WITHOUT IT THE CACHE BELOW CACHED NOTHING. # # A released mcpp is self-contained: with no `MCPP_HOME`, `mcpp self env` @@ -55,7 +77,9 @@ jobs: # (319 MB) -- and a cold cache downloads all of them before the first # compile. The cache key includes the fixtures' manifests, so a run that # changes one of them is always the cold case. - timeout-minutes: 90 + # + # 150 while `MCPP_SOURCE_REF` builds mcpp from source ahead of every step. + timeout-minutes: 150 steps: - uses: actions/checkout@v4 @@ -87,6 +111,36 @@ jobs: echo "MCPP=$MCPP" >> "$GITHUB_ENV" echo "MCPP_VENDORED_XLINGS=$MCPP_VENDORED_XLINGS" >> "$GITHUB_ENV" + # THE ENGINE THE STEPS BELOW RUN: the release fetched above, or mcpp built + # from `MCPP_SOURCE_REF` (see the workflow's `env`). A cold build of mcpp + # is the longest step of the job, which is what its timeout is sized for. + - name: the mcpp under review, built from MCPP_SOURCE_REF when it is set + timeout-minutes: 45 + run: bash .github/scripts/mcpp-under-review.sh + + # THE CRITERION FOR THE STEP ABOVE, IN A LATER STEP, because what that step + # writes to `GITHUB_ENV` governs only the steps after it. + - name: the steps below run the mcpp under review + if: env.MCPP_SOURCE_REF != '' + timeout-minutes: 5 + run: | + set -eu + [ "$MCPP" = "$MCPP_UNDER_REVIEW" ] || { + echo "::error::the steps run $MCPP, and the mcpp built from $MCPP_SOURCE_REF is $MCPP_UNDER_REVIEW"; exit 1; } + version=$("$MCPP" --version | head -1) + [ "$version" = "$MCPP_UNDER_REVIEW_VERSION" ] || { + echo "::error::$MCPP prints '$version', and the build printed '$MCPP_UNDER_REVIEW_VERSION'"; exit 1; } + # A reading, not a criterion: whether each engine declares `pack + # --toolchain` (mcpp#634, E14), which separates the development branch + # from 2026.9.14.1 while both print one version. + for engine in "$MCPP" "$MCPP_RELEASED"; do + "$engine" pack --help > pack-help.txt 2>&1 || true + if grep -q -- '--toolchain' pack-help.txt; then t=yes; else t=no; fi + echo "READING $engine: $("$engine" --version | head -1), pack --toolchain: $t" + done + rm -f pack-help.txt + echo "ok: the steps run $version, built from $MCPP_SOURCE_REF" + # The rule's output is a header holding the SPIR-V module; the program # checks the magic number in its first word, so no Vulkan runtime is # needed and what is tested is the rule and the engine path feeding it. @@ -1123,6 +1177,14 @@ jobs: working-directory: tests/ios-app-consumer run: MCPP="$MCPP" ./check-ios-plan.sh + # dist-apple's framework directory, signature, runner and disk image, as + # planned for the macOS and iOS rows (mcpp#634, B1 to B3). The same + # fixture is built, signed, run and imaged on the macOS runner. + - name: dist-apple's framework, signature, runner and disk image, checked at the plan level + timeout-minutes: 15 + working-directory: tests/app-framework-consumer + run: MCPP="$MCPP" ./check-apple-plan.sh + # dist-web (#622 B3), end to end, the real `mcpp pack --format web # --target wasm32-emscripten` -- see `tests/web-consumer/ # check-web-plan.sh`'s own header for the host-toolchain defect this @@ -1223,23 +1285,20 @@ jobs: working-directory: tests/apk-consumer run: MCPP="$MCPP" ./check-apk-features.sh - # A dependency linked as a shared object on the Android row reaches - # `lib//` beside the app's own: walked from the app's NEEDED at - # command time, because the closure on this row is `not-walked` and the - # engine stages the app's object and the deployed files only (0.9.3). - - name: dist-apk carries the graph's shared libraries + # THE CLOSURE THE ENGINE STAGED (mcpp#634, B5). A dependency linked as a + # shared object on the Android row reaches `lib//` beside the app's + # own because the engine stages the closure and names it in the stage + # manifest (2026.9.14.2+), and the member reads that tree. Five criteria: + # two packs in a row carry the dependency's library both times; two + # triples give one signed APK listing both ABIs; `--format aab` gives an + # App Bundle bundletool validates and jarsigner verifies, and a universal + # APK built from it; a refusal prints its reason; and a stage manifest + # without `needs` lines is refused naming the engine floor. The script's + # header records what 0.9.3 did for each. + - name: dist-apk reads the staged closure, packs two ABIs into one APK, and builds an App Bundle + timeout-minutes: 30 working-directory: tests/apk-consumer-shared - run: | - "$MCPP" build --target x86_64-linux-android - "$MCPP" pack --format apk --target x86_64-linux-android | tee pack.log - apk=$(find target -name 'apk-consumer-shared.apk' | head -1) - test -n "$apk" || { cat pack.log; echo "FAIL: no apk-consumer-shared.apk"; exit 1; } - JDK=$(find "$MCPP_HOME/registry/data/xpkgs/xim-x-jdk-temurin" -mindepth 1 -maxdepth 1 -type d | head -1) - "$JDK/bin/jar" tf "$apk" | tee listing.log - for f in "lib/x86_64/libapk-consumer-shared.so" "lib/x86_64/libapk-consumer-dep.so"; do - grep -qxF "$f" listing.log || { echo "FAIL: the apk does not list $f"; exit 1; } - done - echo "ok: the dependency's shared object is in the apk beside the app's" + run: MCPP="$MCPP" ./check-apk-closure.sh # Compiles the device unit on a machine with no GPU: the clang route # produces sm_89 code from the payload toolkit. Running it needs a @@ -1498,7 +1557,8 @@ jobs: rules-cross-platform: name: rules (${{ matrix.name }}) runs-on: ${{ matrix.runs-on }} - timeout-minutes: 60 + # 150 while `MCPP_SOURCE_REF` builds mcpp from source ahead of every step. + timeout-minutes: 150 strategy: fail-fast: false matrix: @@ -1593,6 +1653,36 @@ jobs: echo "MCPP=$MCPP" >> "$GITHUB_ENV" echo "MCPP_VENDORED_XLINGS=$MCPP_VENDORED_XLINGS" >> "$GITHUB_ENV" + # THE ENGINE THE STEPS BELOW RUN: the release fetched above, or mcpp built + # from `MCPP_SOURCE_REF` (see the workflow's `env`). A cold build of mcpp + # is the longest step of the job, which is what its timeout is sized for. + - name: the mcpp under review, built from MCPP_SOURCE_REF when it is set + timeout-minutes: 45 + run: bash .github/scripts/mcpp-under-review.sh + + # THE CRITERION FOR THE STEP ABOVE, IN A LATER STEP, because what that step + # writes to `GITHUB_ENV` governs only the steps after it. + - name: the steps below run the mcpp under review + if: env.MCPP_SOURCE_REF != '' + timeout-minutes: 5 + run: | + set -eu + [ "$MCPP" = "$MCPP_UNDER_REVIEW" ] || { + echo "::error::the steps run $MCPP, and the mcpp built from $MCPP_SOURCE_REF is $MCPP_UNDER_REVIEW"; exit 1; } + version=$("$MCPP" --version | head -1) + [ "$version" = "$MCPP_UNDER_REVIEW_VERSION" ] || { + echo "::error::$MCPP prints '$version', and the build printed '$MCPP_UNDER_REVIEW_VERSION'"; exit 1; } + # A reading, not a criterion: whether each engine declares `pack + # --toolchain` (mcpp#634, E14), which separates the development branch + # from 2026.9.14.1 while both print one version. + for engine in "$MCPP" "$MCPP_RELEASED"; do + "$engine" pack --help > pack-help.txt 2>&1 || true + if grep -q -- '--toolchain' pack-help.txt; then t=yes; else t=no; fi + echo "READING $engine: $("$engine" --version | head -1), pack --toolchain: $t" + done + rm -f pack-help.txt + echo "ok: the steps run $version, built from $MCPP_SOURCE_REF" + # FIRST, BECAUSE IT IS THE CHEAPER QUESTION AND THE MORE INFORMATIVE # ANSWER. Every other step here drives one rule end to end and needs # that rule's payload; this one names no accelerator, downloads nothing, @@ -1787,6 +1877,15 @@ jobs: exit 1; } echo "ok: the MSI installs msi-consumer.exe, $insidesize bytes, byte-for-byte the linked program" + # `--format setup` (mcpp#634, B4): a Burn bundle with WiX's stock + # bootstrapper application, chaining the MSI the step above measured, and + # built through the extension `xim:wix` 5.0.2-1 carries. + - name: dist-wix produces a Burn bundle that chains the MSI + if: runner.os == 'Windows' + timeout-minutes: 20 + working-directory: tests/msi-consumer + run: MCPP="$MCPP" ./check-setup.sh + - name: dist-apple produces a bundle that launches if: runner.os == 'macOS' working-directory: tests/app-consumer @@ -1834,6 +1933,27 @@ jobs: grep -q 'closure = not-walked' pack.log || echo "note: the stage manifest's closure line was not echoed by pack" echo "ok: one bundle, a valid plist, it launches, and the resource is where NSBundle looks" + # THE CLOSURE AS A FRAMEWORK, ON THE BUNDLE (mcpp#634, B1 to B3). The + # plan-level half runs on Linux. This half reads the linked program's + # load commands, verifies the bundle's ad-hoc signature, runs the program + # with and without its framework, runs the bundle through `mcpp run + # --format app` with no runner in the manifest, and attaches the image. + - name: dist-apple carries the closure as a signed framework, runs through macapp-run, and writes a disk image + if: runner.os == 'macOS' + timeout-minutes: 30 + working-directory: tests/app-framework-consumer + run: MCPP="$MCPP" ./check-apple-bundle.sh + + # rules-metal (mcpp#634, B6): three Metal libraries from two shaders, and + # the compiler's dependency file in the graph. On a runner without the + # Metal toolchain the script asserts the rule's refusal and annotates the + # run with a warning that the two criteria are unmeasured. + - name: rules-metal compiles Metal libraries, or records that this runner has no Metal toolchain + if: runner.os == 'macOS' + timeout-minutes: 20 + working-directory: tests/metal-consumer + run: MCPP="$MCPP" ./check-metal.sh + # THE iOS ROW, FOR REAL (#622 B1's other half). `tests/ios-app-consumer` # is the same fixture the Linux `consumers` job checks at the plan # level (see that fixture's own header); here it is built, packed and @@ -1908,22 +2028,28 @@ jobs: # THE SIMULATOR, THROUGH THE DECLARED RUNNER. `mcpp run --format # app` hands the packaged bundle to `runner = ["simctl-run"]` - # (`xim:apple-simulator-tools` 0.2.0, declared on this target row), - # which installs and launches it rather than spawning a bare - # executable -- the branch that package's 0.2.0 header records as - # needed for an installed bundle. This is the same path a - # consumer's own `mcpp run --target aarch64-ios-sim --format app` - # takes, not a bare `simctl spawn` this step could call directly. + # (`xim:apple-simulator-tools` 0.3.0, declared on this target row), + # which installs the bundle and, because its executable does not + # load UIKit, runs the installed executable with `simctl spawn`. This + # is the same path a consumer's own `mcpp run --target aarch64-ios-sim + # --format app` takes, not a bare `simctl spawn` this step could call + # directly. + # + # THE PROGRAM EXITS 7, AND 7 IS THE CRITERION. `simctl launch`, which + # 0.2.0 used, returns 0 for an application that exits 7, so a fixture + # that exited 0 could not tell a runner that returns the status from + # one that loses it (mcpp#634's triage record, section 7.2). out=$("$MCPP" run --target aarch64-ios-sim --format app 2>&1) && rc=0 || rc=$? printf '%s\n' "$out" | tail -20 - [ "$rc" -eq 0 ] \ - || { echo "FAIL: mcpp run --target aarch64-ios-sim --format app exited $rc"; exit 1; } - # `simctl launch --console-pty` returns the program's output through - # a pty, so a line may end in a carriage return; the marker is asserted - # as a line after that is stripped. + [ "$rc" -eq 7 ] \ + || { echo "FAIL: mcpp run --target aarch64-ios-sim --format app exited $rc, and the program exits 7"; exit 1; } + # A line may end in a carriage return when the output passes through + # a pty, so the marker is asserted as a line after that is stripped. + # The diagnostic uses `od -c`, which BSD and GNU both accept; BSD + # `cat` refuses `-A`. tr -d '\r' <<<"$out" | grep -Eq '^[[:space:]]*1-2-3[[:space:]]*$' \ - || { echo "FAIL: the program's output line '1-2-3' is absent"; printf '%s\n' "$out" | cat -A | tail -8; exit 1; } - echo "ok: a flat iOS Simulator bundle, MinimumOSVersion 17.0, and the simulator ran it" + || { echo "FAIL: the program's output line '1-2-3' is absent"; printf '%s\n' "$out" | tail -8 | od -c; exit 1; } + echo "ok: a flat iOS Simulator bundle, MinimumOSVersion 17.0, and the simulator ran it and returned its status 7" - name: the rule declared its own compiler working-directory: tests/spirv-consumer diff --git a/README.md b/README.md index 08bab32..d24afa2 100644 --- a/README.md +++ b/README.md @@ -66,16 +66,17 @@ engine's own module family and is not used here. | `rules-ascendc` | `mcpp.rules.ascendc` | 2026.9.6.6 | `[build] accel = "ascend8.5+{dav-c220}"`, a constrained glob for `*.asc`. Compiles with BiSheng in MIXED mode, so the object carries the device binary and a host-callable launcher and joins the ordinary link -- no registration file and no device-link step. Its own engine needs are `.asc` in the device-source table and `mcpp::link_flag` for the `-rpath-link` the toolkit's shared libraries require, both 2026.9.6.5 | | `rules-cuda` | `mcpp.rules.cuda` | 2026.9.6.6 | `[build] accel = "cuda…"`, a constrained glob for `*.cu`; the clang route with an LLVM toolchain, the nvcc route with a GCC one | | `rules-hip` | `mcpp.rules.hip` | 2026.9.6.6 | `[build] accel = "hip, cuda12.9+{sm_89}"`, a constrained glob for `*.hip`. On the NVIDIA platform HIP is a header layer over the CUDA runtime, so the compiler is the project's own clang and there is no ROCm on the machine | +| `rules-metal` | `mcpp.rules.metal` | 2026.9.8.1 | the Metal toolchain of the macOS host's Xcode, located rather than installed: Xcode is not redistributable, so no payload is declared. `.metal` sources the project names on a macOS or iOS row become one `xcrun --sdk metal` action per shader (`-MMD`, so an edited `#include` recompiles the shaders that include it) and one `xcrun --sdk metallib` action per library, placed beside the program with `mcpp::deploy` under `metallib/`, which `dist-apple` maps into the bundle's resources. `compile(shaders)` compiles one source several times with definitions of its own, one library per `shader`; `options::library` links every shader into one library (`default` is the one `newDefaultLibrary` finds). Before planning anything the rule asks `xcrun --sdk --show-sdk-path` and `--find metal` / `--find metallib`, and refuses naming the command that answered nothing, because a missing SDK and a missing compiler have different remedies (Xcode 26 installs the Metal toolchain as a separate component). A shader on any other row is refused naming the row. CI compiles the fixture on `macos-15` and checks each library's magic, and that a header edit recompiles only the shaders that include it | | `rules-slang` | `mcpp.rules.slang` | 2026.9.7.1 | `[build] accel = "vulkan1.2"`, a constrained glob for `*.slang`. Slang is a different language from GLSL rather than a second driver for it -- its own module system, generics, and targets beyond SPIR-V -- so it is a rule of its own. `.slang` is **not** in the engine's device-source table: this feature declares `device_extensions = [".slang"]` and `rule_module = "mcpp.rules.slang"`, and the engine routes it from there. That is the criterion for the whole arrangement -- a new device language costs no engine release. Since 0.7.0 it has the same `options::storage` axis as `rules-spirv` (header / object / sidecar), `options::extra_args` for the arguments the rule has no field for, and `options::per_file` for what one shader gets that the others do not -- a project with a `-fvk-use-gl-layout` and one shader needing `-emit-spirv-via-glsl` writes both without leaving one `compile()` call | | `rules-spirv` | `mcpp.rules.spirv` | 2026.9.6.6 | `[build] accel = "vulkan1.2"`, a constrained glob for the shader stages; compiles each shader through a `role = "source"` action and states which of the two compilers produced it | | `rules-sycl` | `mcpp.rules.sycl` | 2026.9.6.6 | `[build] accel = "sycl"` or `"sycl, cuda12.9+{sm_89}"`, a constrained glob for `*.sycl`, and `compat:sycl-runtime` so the artifact can reach `libsycl.so.9` at run time. Its own engine need is `.sycl` in the device-source table, 2026.9.6.1 | | `tools-embed` | `mcpp.tools.embed` | 2026.9.5.4 | nothing beyond mcpp: it reads a file and writes a header while the build program runs. The floor is the release whose fast path compares a declared file input, without which an edit to the data does not reach the binary | | `tools-island` | `mcpp.tools.island` | 2026.9.7.1 | nothing beyond mcpp: it reads marked entry points out of an island's own source and writes the `extern "C"` boundary header its compiler reads and the module the C++ side imports. Not a device rule -- it claims no extension, and a project calls it from its own `build.mcpp` | | `dist-appimage` | `mcpp.dist.appimage` | 2026.9.11.1 | `xim:appimagetool`, which this feature declares on the `cfg(linux)` axis. Linux only. Turns the tree `mcpp pack` staged into one AppImage: the staged bundle is already an AppDir bar three files, so the member writes an `AppRun`, a `.desktop` entry and an icon into it and invokes one tool -- it never copies or re-lays-out a tree that can be hundreds of megabytes | -| `dist-wix` | `mcpp.dist.wix` | 2026.9.11.1 | `xim:wix`, which this feature declares on the Windows target axis; the .NET 6 runtime the tool needs is a Windows component the payload does not carry, and `wix --version` names it when it is missing. Windows only. Renders a `.wxs` and passes the program in as a preprocessor variable, because a bind path that resolves to nothing is silent | -| `dist-apple` | `mcpp.dist.apple` | 2026.9.11.2 (macOS), 2026.9.12.3 (iOS) | the base macOS install (`ditto`, and `codesign` only when an identity is given). macOS: `Contents/`-shaped, as always. iOS (`aarch64-ios-sim`, `aarch64-ios`): a flat bundle at the same call site -- no separate feature, no separate module -- with `MinimumOSVersion` from `mcpp::min_platform_version()` (#622 A11), `CFBundleSupportedPlatforms` read from `env == "sim"`, `UIDeviceFamily`, `LSRequiresIPhoneOS`, and a directory of flat PNGs listed under `CFBundleIcons` in place of macOS's single `.icns` file. Signing is skipped on the simulator row (`options::identity` is ignored, with a `mcpp::warning` naming why) and unchanged on the device row. The iOS row is measured end to end on `macos-15`: a real `mcpp build`, `mcpp pack --format app` and `mcpp run` against `aarch64-ios-sim`, through `xim:apple-simulator-tools`' `simctl-run`. **The macOS floor is one release higher than its siblings** and the reason is not this member: under 2026.9.11.1 `mcpp pack` staged before dispatching and let a staging failure fail the command, so on a Mach-O program -- which the built-in closure walk refuses, because it uses `LD_TRACE_LOADED_OBJECTS` and dyld answers that by running the program -- every dispatched format was unreachable, including one that reads no staged tree. 2026.9.11.2 makes staging a service to the provider. From 0.9.2 the staged tree's deployed files (`bin//...`, which the engine stages for a Mach-O program before the closure walk since the release for mcpp#630) land at the bundle's resource destination -- `Contents/Resources//...` on macOS, the bundle root on iOS -- and the launcher alone goes to the executable directory, so `CFBundleExecutable` names a file that is where it says. The iOS fixture declares `llvm.libcxx` and `llvm.compiler-rt-builtins` under `cfg(os = "ios")`, which is what an application that imports `std` on those rows declares | +| `dist-wix` | `mcpp.dist.wix` | 2026.9.11.1 | `xim:wix` 5.0.2-1, which this feature declares on the Windows target axis; the .NET 6 runtime the tool needs is a Windows component the payload does not carry, and `wix --version` names it when it is missing. Windows only. `--format msi` renders a `.wxs` and passes the program in as a preprocessor variable, because a bind path that resolves to nothing is silent. From 0.10.0 `--format setup` is a Burn bundle chaining that MSI, with WiX's stock bootstrapper application (`bal:WixStandardBootstrapperApplication`, theme `hyperlinkLicense`, `options::license_url` its link) loaded through the `WixToolset.BootstrapperApplications.wixext` extension the 5.0.2-1 payload carries (`options::extension` names another). The bundle is written as `-.exe` beside the MSI, with an UpgradeCode of its own, and `options::bundle_output` naming `setup.exe` is refused before `wix` runs, because `wix` refuses that name (WIX0388). A project with its own bootstrapper application supplies `options::bundle_wxs`, which receives the MSI as `$(Msi)`. CI builds the bundle on `windows-2022` and compares the MSI `wix burn extract` takes out of it with the one the first action wrote | +| `dist-apple` | `mcpp.dist.apple` | 2026.9.14.2 (0.10.0); 2026.9.11.2 (macOS) and 2026.9.12.3 (iOS) before it | the base macOS install (`ditto`, `codesign`, `hdiutil`), and `xim:macapp-run` for `mcpp run` on macOS, which this feature declares with `when = "run"`. macOS: `Contents/`-shaped, as always. iOS (`aarch64-ios-sim`, `aarch64-ios`): a flat bundle at the same call site -- no separate feature, no separate module -- with `MinimumOSVersion` from `mcpp::min_platform_version()` (#622 A11), `CFBundleSupportedPlatforms` read from `env == "sim"`, `UIDeviceFamily`, `LSRequiresIPhoneOS`, and a directory of flat PNGs listed under `CFBundleIcons` in place of macOS's single `.icns` file. Signing is skipped on the simulator row (`options::identity` is ignored, with a `mcpp::warning` naming why), and the device row signs only with an identity. The iOS row is measured end to end on `macos-15`: a real `mcpp build`, `mcpp pack --format app` and `mcpp run` against `aarch64-ios-sim`, through `xim:apple-simulator-tools`' `simctl-run`. **The macOS floor is one release higher than its siblings** and the reason is not this member: under 2026.9.11.1 `mcpp pack` staged before dispatching and let a staging failure fail the command, so on a Mach-O program -- which the built-in closure walk refuses, because it uses `LD_TRACE_LOADED_OBJECTS` and dyld answers that by running the program -- every dispatched format was unreachable, including one that reads no staged tree. 2026.9.11.2 makes staging a service to the provider. From 0.9.2 the staged tree's deployed files (`bin//...`, which the engine stages for a Mach-O program before the closure walk since the release for mcpp#630) land at the bundle's resource destination -- `Contents/Resources//...` on macOS, the bundle root on iOS -- and the launcher alone goes to the executable directory, so `CFBundleExecutable` names a file that is where it says. The iOS fixture declares `llvm.libcxx` and `llvm.compiler-rt-builtins` under `cfg(os = "ios")`, which is what an application that imports `std` on those rows declares From 0.10.0, with mcpp 2026.9.14.2: the dylibs the engine stages beside a Mach-O program, which the stage manifest's `needs` lines name, go to `Contents/Frameworks/` (`Frameworks/` on iOS) and not to the resources; the program is linked with the rpath that finds them there (`@executable_path/../Frameworks`, `@executable_path/Frameworks` on iOS) through `mcpp::link_flag`, so no file is edited after the link; a macOS bundle without `options::identity` is signed ad hoc, frameworks first and the bundle second, which `codesign --verify --deep --strict` requires of a bundle that carries a framework; an incomplete closure is a `mcpp::warning` naming the unresolved libraries; every refusal is a `mcpp::warning` as well, because the engine discards a build program's output when it exits 0. On macOS the member supplies the runner named `app` (`macapp-run`), so `mcpp run --format app` runs the bundle's executable in the foreground and returns its status with no runner in the manifest; a manifest runner of that name wins. `--format dmg` stages the bundle beside an `Applications` link and writes a UDZO image with `hdiutil create` (`options::volume_name`, `options::dmg`); it is refused on iOS. An engine below 2026.9.14.2 stages no `needs` lines, so the bundle carries no framework, anchors the rpath to the package directory, and hands the bundle directory to the kernel under `mcpp run --format app` unless `--runner app` is typed. CI measures the bundle on `macos-15`: the load command, the signature, the program with and without its framework (exit 7, then "Library not loaded"), `mcpp run --format app` with and without `--runner app`, and `hdiutil verify` and an attached image | | `dist-web` | `mcpp.dist.web` | 2026.9.13.1, the release that carries `${mcpp.self}` and `mcpp stage`'s argument shape as an engine contract (`stage --verify content --output `) -- what lets this member's copy run on every host mcpp does, Windows included, in place of the `cp` this member used through 0.8.0 | nothing beyond mcpp: `wasm32-emscripten` only. Copies `${mcpp.stage_dir}/bin/` -- the `.js` launcher, the implicit `.wasm`, the `.data` when present, and every `mcpp::deploy`'d file, all of which #622 A5 and A4 already stage there -- to `/web/`, dropping the `bin/` prefix a browser has no use for, and writes an `index.html` rendered from a project template or a built-in default that loads the script with a plain `