From 1f4285222d961baccbce67600aa233f45dc731da Mon Sep 17 00:00:00 2001 From: ewowi Date: Fri, 13 Nov 2026 09:06:41 +0100 Subject: [PATCH 1/5] A desktop updates itself through the Firmware module, as a board does MoonLight on Linux, macOS and Windows now installs a new version of itself through the same Firmware card and routes as an ESP32: it fetches or receives the release's program, checks that it is a program for this system and processor (and its SHA-256 when given), keeps the old one as MoonLight.prev and restarts on the new one in place. The Debian package installs MoonLight as a confined systemd service that owns its program, so it updates itself too, and rolls back to the previous program when a new one keeps failing in its first minute. KPI: 256lights | Desktop:2544KB | tick:8/14/13/3/8/11/294/8/8/10us(FPS:125000/71428/76923/333333/125000/90909/3401/125000/125000/100000) | ESP32:2316KB | src:393(99476) | test:292(70125) | functions-over:333 Commit: src +768/-250 lines | test +386/-10 lines | code findings 1607 -> 1607 (complex function -2, large stack frame +2) | duplicated lines 3186 -> 3186 (+0) **Core** - platform: UpdateMode (self, host, manual) replaces hasOta; a container is updated by its host, a native program in a folder it can write updates itself, otherwise by hand. /api/system and the Firmware module's hidden `update` control carry it. - Desktop otaBegin/otaWrite/otaEnd write MoonLight.next beside the program; otaEnd vets the header (ProgramImage.h: ELF, Mach-O, PE) and the digest, keeps .prev, renames the new program in and marks it .pending; the new program confirms itself after kBootStableMs. reboot() restarts in place: execv on POSIX (every socket close-on-exec where it is made), CreateProcessW in the same console on Windows, retried. - One HTTP client: httpDownload streams with redirects (https only), the size and failure status, through curl and WinHTTP alike. - HttpServerModule: the firmware routes gate on updateMode; /api/firmware/url passes the sha256; installThroughMoonBase and readImageUrl extracted; image URLs are capped at 255 characters, the MoonBase handoff's limit, and refused with 400 past it; sendError replaces four hand-built error answers. - The firmware name desktop-- comes from CMake. **UI** - The Firmware card and the install picker take the release's firmware-desktop-- asset with its digest, uniform with a board's; a host- or hand-updated desktop shows "by hand". **Scripts/MoonDeck** - package_desktop.py stages firmware-desktop---v.bin; the .deb seeds /usr/lib/moonlight/MoonLight and installs moonlight.service (own user, StateDirectory, Restart=always, OnFailure rollback, NoNewPrivileges, CAP_NET_RAW alone for ColorLight's raw frames, ProtectSystem=strict) with postinst, prerm, postrm and the rollback script. - release.yml builds the desktop programs on pull requests, as artifacts only; release_stage.py maps a PR ref to a -dev version. - run_live_scenario.py no longer relaunches the desktop itself. **Tests** - unit_platform_update: install, refusal, abandon, digest in either case, read-only folder, the targets a desktop refuses. unit_ProgramImage: each format and processor. A 300-character image URL is refused with 400. test_package_deb: layout and confinement. ui-update-asset: the uniform asset name. - Desktop 3268/3268, JS 396, Python 765. On this Mac: /api/system says desktop-macos-arm64, update self; /api/reboot restarts in place (same PID). **Docs/CI** - installing-on-linux: the service, its rights and updates; MIGRATING v7.1.0 entry; openapi (update, sha256, the 255-character URL cap); mooncore, testing, coding-standards (Windows builds on PRs), small-boards; 02-install shrunk; the plan's step 1 done, step 2 (the map) next. - large stack frame 31 -> 33: handleFirmwareUrl and handleImageUrl had these frames on every board all along; the desktop, where frames are measured, compiled their bodies out until now. The URL cap and sendError shrank one of them; the rest is helpers the compiler inlines. **Reviews** - Reviewer, host_update.h included only under the macOS screen guard, breaking Linux, Windows and GCC: done, the GCC build passes. - Reviewer, exiting to systemd counts reboots against StartLimit and INVOCATION_ID can be inherited: done, reboot() always restarts in place on POSIX. - Reviewer, the service unconfined: done, systemd.exec confinement pinned in a test. - Reviewer, an https -> http redirect and no digest from the Firmware card: done, redirects stay on https and the card posts the digest. - Reviewer, a failed CreateProcessW exits silently: done, retried 5 s, then the reason. - Reviewer, the fetch thread writing shared progress globals: deferred, the same shape as the ESP32 update task; atomics change the cross-platform interface and are their own change. - Reviewer, three stale comments: done. - Reviewer, a literal behind a const_cast: done, the control binds a documented member. - Reviewer, the read-only test passing as root: done, that half skips under root. - Reviewer, ColorLight from the service needs CAP_NET_RAW: done, granted as the one capability. - MoonBase freshness: SKIP, moonbase/ untouched. --no-jit build: SKIP, no MoonLive source touched. Improv smoke test: SKIP, no Improv code touched. taglines: SKIP, README and CLAUDE.md untouched. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/release.yml | 16 +- CMakeLists.txt | 21 + Dockerfile | 4 +- docs/contributing/coding-standards.md | 2 +- docs/design/small-boards.md | 2 +- docs/explanation/architecture/mooncore.md | 2 +- docs/help/help.json | 3 + docs/reference/MIGRATING.md | 26 ++ docs/reference/installing-on-linux.md | 24 +- docs/reference/metrics/code.json | 4 +- docs/reference/metrics/docgen.json | 6 +- docs/reference/metrics/prose.json | 8 +- docs/reference/metrics/repo-health.json | 42 +- docs/reference/metrics/repo-health.md | 92 ++--- docs/reference/openapi.yaml | 13 +- docs/reference/testing.md | 2 +- ...9 - Devices, groups and the network map.md | 48 ++- moondeck/ci/check_glibc_floor.py | 2 +- moondeck/ci/linux/moonlight-rollback.service | 7 + moondeck/ci/linux/moonlight.service | 34 ++ moondeck/ci/linux/postinst | 23 ++ moondeck/ci/linux/postrm | 9 + moondeck/ci/linux/prerm | 6 + moondeck/ci/linux/rollback | 12 + moondeck/ci/package_desktop.py | 46 ++- moondeck/ci/release_stage.py | 9 +- moondeck/moondeck_ui/app.js | 8 +- moondeck/scenario/run_live_scenario.py | 35 +- moonwalk/02-install.yaml | 36 +- src/core/system/FirmwareUpdateModule.h | 5 + src/core/system/HttpServerModule.cpp | 118 +++--- src/core/system/HttpServerModule.h | 8 + src/core/util/ProgramImage.h | 74 ++++ src/core/util/build_info.h | 2 +- src/core/util/sha256.h | 24 +- src/platform/desktop/host_update.h | 32 ++ src/platform/desktop/main_desktop.cpp | 2 + src/platform/desktop/platform_config.h | 3 - src/platform/desktop/platform_desktop.cpp | 145 ++++--- .../desktop/platform_desktop_update.cpp | 384 ++++++++++++++++++ src/platform/esp32/platform_config.h | 3 - src/platform/esp32/platform_esp32.cpp | 3 + src/platform/esp32/platform_esp32_ota.cpp | 2 +- src/platform/platform.h | 15 +- src/platform/teensy/platform_config.h | 3 - src/platform/teensy/platform_teensy.cpp | 2 + src/platform/teensy/platform_teensy_ota.cpp | 18 +- src/ui/app.js | 122 +++--- src/ui/install-picker.js | 53 +-- test/CMakeLists.txt | 2 + test/js/installer-desktop-download.test.mjs | 12 +- test/js/ui-update-asset.test.mjs | 52 +++ test/python/test_package_deb.py | 63 +++ test/python/test_release_stage.py | 4 + test/unit/core/unit_FirmwareUpdateModule.cpp | 9 +- .../unit/core/unit_HttpServerModule_apply.cpp | 4 +- .../core/unit_HttpServerModule_routes.cpp | 16 + test/unit/core/unit_ProgramImage.cpp | 70 ++++ test/unit/core/unit_platform_update.cpp | 164 ++++++++ 59 files changed, 1520 insertions(+), 436 deletions(-) create mode 100644 moondeck/ci/linux/moonlight-rollback.service create mode 100644 moondeck/ci/linux/moonlight.service create mode 100755 moondeck/ci/linux/postinst create mode 100755 moondeck/ci/linux/postrm create mode 100755 moondeck/ci/linux/prerm create mode 100755 moondeck/ci/linux/rollback create mode 100644 src/core/util/ProgramImage.h create mode 100644 src/platform/desktop/host_update.h create mode 100644 src/platform/desktop/platform_desktop_update.cpp create mode 100644 test/js/ui-update-asset.test.mjs create mode 100644 test/python/test_package_deb.py create mode 100644 test/unit/core/unit_ProgramImage.cpp create mode 100644 test/unit/core/unit_platform_update.cpp diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 597f92730..d3708fe48 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -43,6 +43,17 @@ on: - 'docs/**' - 'mooninstaller/**' - 'mkdocs.yml' + # A pull request builds the desktop programs and uploads them as workflow artifacts, to install through a device's Firmware card before the + # merge; the release, container and Pages jobs run on main and tags only, so a pull request publishes nothing. The ESP32 matrix is built + # locally and the Teensy by test.yml, so they stay off pull requests. + pull_request: + paths: + - 'src/**' + - 'test/**' + - 'moondeck/ci/**' + - 'CMakeLists.txt' + - 'library.json' + - '.github/workflows/release.yml' workflow_dispatch: inputs: tag: @@ -58,7 +69,8 @@ on: # their own concurrency group so a tag and a branch can build in parallel. concurrency: group: release-${{ github.ref }} - cancel-in-progress: false + # A newer push to a pull request replaces its running build; a release always finishes. + cancel-in-progress: ${{ github.event_name == 'pull_request' }} # Minimal default permissions for every job (read-only). The release job # overrides this locally with the write/id-token scopes it needs to publish @@ -101,6 +113,7 @@ jobs: # standard bridge. This is the ONLY firmware list in CI, and the chip is the IDF target. firmwares: needs: verify-version + if: github.event_name != 'pull_request' # and build-esp32 with it, which needs this list runs-on: ubuntu-latest outputs: list: ${{ steps.gen.outputs.list }} @@ -432,6 +445,7 @@ jobs: # The Teensy 4.1 firmware, a HEX for teensy_loader_cli or PJRC's Teensy app; test.yml builds it on every PR as well. build-teensy: needs: verify-version + if: github.event_name != 'pull_request' # test.yml builds the Teensy firmwares on a pull request runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 diff --git a/CMakeLists.txt b/CMakeLists.txt index bf7117ded..ada2ce00e 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -172,6 +172,26 @@ endif() if(MM_VERSION) set_property(SOURCE src/core/util/build_info.cpp APPEND PROPERTY COMPILE_DEFINITIONS MM_VERSION="${MM_VERSION}") endif() +# A desktop build names its firmware as the release names its program, firmware-desktop---v.bin (package_desktop.py), +# so the Firmware card, the map and Home Assistant install it as they install a board, which takes its name from esp32/main/CMakeLists.txt. +if(APPLE) + set(MM_DESKTOP_OS macos) +elseif(WIN32) + set(MM_DESKTOP_OS windows) +else() + set(MM_DESKTOP_OS linux) +endif() +string(TOLOWER "${CMAKE_SYSTEM_PROCESSOR}" MM_DESKTOP_CPU) +if(MM_DESKTOP_CPU MATCHES "^(arm64|aarch64)$") + set(MM_DESKTOP_CPU arm64) +elseif(MM_DESKTOP_CPU MATCHES "^(x86_64|amd64)$") + set(MM_DESKTOP_CPU x64) +else() + unset(MM_DESKTOP_CPU) # a processor the release packages no program for: the name stays unknown +endif() +if(MM_DESKTOP_CPU) + set_property(SOURCE src/core/util/build_info.cpp APPEND PROPERTY COMPILE_DEFINITIONS MM_FIRMWARE_NAME="desktop-${MM_DESKTOP_OS}-${MM_DESKTOP_CPU}") +endif() # Platform library (desktop). moonlive_emit.cpp is the desktop MoonLive backend (host-ISA # codegen) — it lives here because emitted machine code is platform/ISA-specific. @@ -179,6 +199,7 @@ add_library(mm_platform src/platform/desktop/platform_desktop.cpp src/platform/desktop/platform_desktop_audio.cpp src/platform/desktop/platform_desktop_hdmi.cpp + src/platform/desktop/platform_desktop_update.cpp src/platform/desktop/moonlive_emit_noarch.cpp src/platform/desktop/moonlive_asm_noarch.cpp src/platform/desktop/moonlive_emit_arm64.cpp diff --git a/Dockerfile b/Dockerfile index f7874688d..d1e2b6527 100644 --- a/Dockerfile +++ b/Dockerfile @@ -84,7 +84,7 @@ RUN apt-get update \ && pulse=$(find /usr/lib -name libpulse.so.0 | head -1) \ && test -n "$pulse" \ # Captured first, so a failing ldd, or one naming a library it cannot find, stops the build here. - && ldd /rootfs/usr/bin/MoonLight > /tmp/deps \ + && ldd /rootfs/usr/lib/moonlight/MoonLight > /tmp/deps \ && ldd "$pulse" >> /tmp/deps \ && ! grep -q "not found" /tmp/deps \ && echo "libpulse.so.0 => $pulse (0)" >> /tmp/deps \ @@ -108,7 +108,7 @@ RUN apt-get update \ # up base updates (docker buildx imagetools inspect gcr.io/distroless/cc-debian13:latest). FROM gcr.io/distroless/cc-debian13@sha256:9b615fff20e1a4fad29c2b30562580b212c7dd5e2225236735cca0070ed11c78 -COPY --from=fetch /rootfs/usr/bin/MoonLight /usr/bin/MoonLight +COPY --from=fetch /rootfs/usr/lib/moonlight/MoonLight /usr/bin/MoonLight # The libraries the loader resolved in the fetch stage, collected there rather than named here. # Without this the container starts and dies immediately on "libcurl.so.4: cannot open shared # object file"; the release runs each image before it publishes one. diff --git a/docs/contributing/coding-standards.md b/docs/contributing/coding-standards.md index 55f07bdae..bf23df2e5 100644 --- a/docs/contributing/coding-standards.md +++ b/docs/contributing/coding-standards.md @@ -183,7 +183,7 @@ All targets build warnings-as-errors: `-Wall -Wextra -Werror` on Clang/GCC (macO **A clean local build is not proof the Windows build passes.** MSVC's `/W4` flags things Clang/GCC's `-Wall -Wextra` don't, most commonly **signed/unsigned mismatch in comparisons** (C4389), e.g. `(x & 1) == 1u` where the left side is signed: harmless logically, fatal under `/WX`. This has slipped past the macOS gate and broken Windows CI more than once. So: - In a comparison, keep both sides the same signedness, don't mix a signed expression with an unsigned literal (`== 1`, not `== 1u`, when the other side is signed). Watch `& `, `%`, and subtraction results, which carry the signedness of their operands. -- A change that only built+passed on macOS/Linux is **not** verified for Windows. The Windows CI job (`release.yml`) is the real gate for MSVC-only warnings; let it run before considering a `src/`-touching change done, or build with MSVC locally if you have it. +- A change that only built+passed on macOS/Linux is **not** verified for Windows. The Windows CI job (`release.yml`, on every pull request touching `src/` as well as on main) is the real gate for MSVC-only warnings; let it run before considering a `src/`-touching change done, or build with MSVC locally if you have it. Diagnostics as you type come from clangd, using the same `.clang-tidy` CI runs; once-per-machine setup is [Set up the editor](../moonwalk.md#setup-editor). diff --git a/docs/design/small-boards.md b/docs/design/small-boards.md index 23c2b09e9..ac6b96af8 100644 --- a/docs/design/small-boards.md +++ b/docs/design/small-boards.md @@ -138,7 +138,7 @@ MoonLight's UI ↔ device link is, at its heart, a **JSON control/state channel* - **MoonLight's portable core + BLE-UI as a showcase: pursue as a time-boxed spike.** The core ports with essentially no architectural work, the small-LED story is a coherent niche rather than a crippling, and, the decisive finding, the **BLE-UI channel is clean and well-precedented**, bandwidth-comfortable, and reuses the transport-agnostic control/state core we already have. The showcase/advertising value is real and lands independent of LED scale. **Gate: acceptable only if the iOS-client gap (Web Bluetooth ≠ Safari) is fine for a demo**, which for an advertisement/proof-of-concept it is (Android + desktop Chrome/Edge reach the device; an iOS companion app is a later, separate call). **Concrete first step if pursued** (sized as a spike, ~1–2 weeks): -1. A `src/platform/nrf/` skeleton under Zephyr on an **nRF52840-DK**: `platform_config.h` with all LED-lane constants 0 except one new PWM/I2S lane flag, `hasNetwork = false`, `hasWiFi/hasEthernet/hasOta = false`; implement the time / heap / fs (LittleFS) / GPIO seam groups. This alone proves the domain-neutral core boots and runs its Scheduler + persistence on Zephyr. +1. A `src/platform/nrf/` skeleton under Zephyr on an **nRF52840-DK**: `platform_config.h` with all LED-lane constants 0 except one new PWM/I2S lane flag, `hasNetwork = false`, `hasWiFi/hasEthernet = false`, `updateMode()` answering manual; implement the time / heap / fs (LittleFS) / GPIO seam groups. This alone proves the domain-neutral core boots and runs its Scheduler + persistence on Zephyr. 2. One **I2S-EasyDMA WS2812 driver** (the whole-frame-DMA shape, closest to `i80Ws2812*`) on a single lane driving a small strip/matrix, the visible output that makes the demo tangible. 3. The **BLE-UI transport**: a GATT custom service with a control-write and a state-notify characteristic, framing the *existing* JSON control/state onto BLE (chunked to MTU), plus a minimal **Web Bluetooth client page** (Android/desktop Chrome) that renders the generic module UI from the notified state and writes control changes back. This is the load-bearing deliverable, it's what turns "the core runs on nRF" into "and you can *use* it from your browser over Bluetooth." diff --git a/docs/explanation/architecture/mooncore.md b/docs/explanation/architecture/mooncore.md index 00f2b7fe6..7755d0e5c 100644 --- a/docs/explanation/architecture/mooncore.md +++ b/docs/explanation/architecture/mooncore.md @@ -104,7 +104,7 @@ Only abstract what you need: - **Memory**: `alloc(size)`, `free(ptr)`. Prefers PSRAM on ESP32, falls back to regular heap; `allocInternal(size)` where only internal RAM will do (see [Internal RAM](#internal-ram)). `freeHeap()`, `maxAllocBlock()` for diagnostics. (`heap_caps_malloc` / `std::malloc`) - **Executable memory**: `allocExec(size)` / `freeExec(ptr, size)` allocate memory the CPU can *fetch and execute* from, and `writeExec(dst, src, len)` copies emitted machine code into it safely. Used by the MoonLive live-script engine (below) to place the native code it compiles. All the W^X and instruction-cache quirks live behind these three functions. On ESP32 that is IRAM via `MALLOC_CAP_EXEC`, with 32-bit-aligned stores and a cache sync so the core fetches fresh code. On POSIX desktops it is an `mmap` `PROT_EXEC` page, with `MAP_JIT` and a write-protect toggle on macOS-arm64. On Windows it is a `VirtualAlloc` `PAGE_EXECUTE_READWRITE` page plus `FlushInstructionCache`. (`heap_caps_malloc(MALLOC_CAP_EXEC)` / `mmap` / `VirtualAlloc`) - **Networking**: `UdpSocket` for ArtNet send. `TcpConnection` / `TcpServer` for HTTP + WebSocket; `TcpConnection::writeSome` is a non-blocking partial write (returns bytes written, 0 = would-block) so a backpressured browser can't stall the render loop. (lwIP sockets / BSD sockets) -- **Scheduling**: `yield()` (cooperative yield to OS/RTOS), `delayMs(ms)` (blocking sleep, off-path only), `delayUs(us)` (a microsecond busy-wait, only for sub-millisecond hardware timing a driver owns, such as the WS2812 300 µs inter-frame latch in `RmtLedDriver`), and `reboot()`. General pacing uses the non-blocking `millis()` gate instead. (`vTaskDelay` / `esp_rom_delay_us` / `esp_restart` on ESP32; `std::this_thread::sleep_for` / `std::exit` on desktop) +- **Scheduling**: `yield()` (cooperative yield to OS/RTOS), `delayMs(ms)` (blocking sleep, off-path only), `delayUs(us)` (a microsecond busy-wait, only for sub-millisecond hardware timing a driver owns, such as the WS2812 300 µs inter-frame latch in `RmtLedDriver`), and `reboot()`. General pacing uses the non-blocking `millis()` gate instead. (`vTaskDelay` / `esp_rom_delay_us` / `esp_restart` on ESP32; `std::this_thread::sleep_for` on desktop, where `reboot()` starts the program again: in place by `execv`, by a new process on Windows) - **Platform config**: `platform_config.h` per platform: compile-time constants like `hasPsram` and `hasWiFi`. Each platform provides its own version; `types.h` includes it without `#ifdef`. Core code branches on these via `if constexpr` (such as NetworkModule drops its WiFi cascade when `hasWiFi` is false), so the dead branch is removed from the binary with no `#ifdef` outside `src/platform/`. Abstractions are added when a concrete implementation needs them, not pre-designed. diff --git a/docs/help/help.json b/docs/help/help.json index f2d5a1828..045c3df25 100644 --- a/docs/help/help.json +++ b/docs/help/help.json @@ -4965,6 +4965,9 @@ "appVersion": { "line": "The installed version." }, + "update": { + "line": "How this device takes a new version: self, host or manual." + }, "moonbaseMismatch": { "line": "The installed version." }, diff --git a/docs/reference/MIGRATING.md b/docs/reference/MIGRATING.md index 94231a60e..c95e69e82 100644 --- a/docs/reference/MIGRATING.md +++ b/docs/reference/MIGRATING.md @@ -31,6 +31,32 @@ For a device still on old firmware (no Backup button yet), the [installer page]( ## v7.1.0 +### A desktop updates itself as a board does, and the Debian package installs a service + +**Action: *nothing* on most desktops; *update a file* on a Linux install from the `.deb` with a systemd unit written from the install guide.** + +A desktop build names its firmware `desktop--`, such as `desktop-linux-arm64`, where it said `unknown`, and installs `firmware--v.bin` from a release through the Firmware card, the Devices map and `/api/firmware/url`, as a board does. `/api/reboot` restarts a desktop where it ended the process: in place, in its terminal window, under systemd or in a container alike. A script that stopped a desktop through `/api/reboot` stops it with Ctrl+C or its service manager instead. + +The package now installs and starts `moonlight.service`, run as its own `moonlight` user from `/var/lib/moonlight/bin`, a folder that user owns, so the Devices map updates it like a board. The program moves from `/usr/bin/MoonLight` to `/usr/lib/moonlight/MoonLight`, the copy the service starts from, and the menu entry opens `http://localhost:8080/`. + +A hand-written `/etc/systemd/system/moonlight.service` overrides the packaged one and names the old path, so remove it before upgrading: + +``` +sudo systemctl disable --now moonlight +sudo rm /etc/systemd/system/moonlight.service +``` + +Settings now live in `/var/lib/moonlight/MoonLight/`. Copy the old user's across once, `pi` here, after the upgrade: + +``` +sudo systemctl stop moonlight +sudo cp -a /home/pi/.local/share/MoonLight/. /var/lib/moonlight/MoonLight/ +sudo chown -R moonlight:moonlight /var/lib/moonlight/MoonLight +sudo systemctl start moonlight +``` + +A Backup taken before the upgrade and restored after it carries them across too. + ### The P4's radio chip status moves from System to the Firmware card **Action: *nothing*, unless a script reads it; read `version` on the Firmware module's WiFi coproc tab instead of `wifiCoproc` on System.** diff --git a/docs/reference/installing-on-linux.md b/docs/reference/installing-on-linux.md index 4a172104c..539a78b65 100644 --- a/docs/reference/installing-on-linux.md +++ b/docs/reference/installing-on-linux.md @@ -12,6 +12,25 @@ The facts follow; the steps are MoonWalk's. [Install MoonLight](../moonwalk.md#i - **The card is a fine home for it.** MoonLight writes to disk only when settings change; the risk is the operating system's own writes. - **HDMI out** works on any machine whose kernel drives its HDMI port, a Pi, a Rockchip board or an Intel or AMD mini PC, through the kernel's display interface (DRM/KMS). Two conditions: no desktop holds the screen, so a Lite or server install, or a desktop install set to boot to the console (`sudo systemctl set-default multi-user.target`), and the user running MoonLight is in the `video` group, which Raspberry Pi OS's first user already is. The first connected HDMI port is the one shown. +## The service + +The package installs MoonLight as `moonlight.service` and starts it, so the machine runs MoonLight from boot on, and the Devices map updates it as it updates a board. + +| | | +|---|---| +| **Program** | `/var/lib/moonlight/bin/MoonLight`, copied from `/usr/lib/moonlight/MoonLight` on every install | +| **User** | `moonlight`, an unprivileged system user in the `video` group for HDMI out | +| **Rights** | Writes its own folder alone and holds one capability, `CAP_NET_RAW`, for the raw Ethernet frames a ColorLight card takes | +| **Config** | `/var/lib/moonlight/MoonLight/.config/` | +| **Logs** | `journalctl -u moonlight` | +| **Page** | `http://localhost:8080/`, which the menu entry opens | + +**Updating, as a board updates.** The service owns the folder its program runs from, so it installs a new version of itself through the same routes, Firmware card and map as a board, from the release's `firmware-desktop-linux--v….bin`. It checks the image is a program for this processor and, from the map, against the SHA-256 GitHub publishes for it; it keeps the running program as `MoonLight.prev`, puts the new one in its place and restarts on it in place, the same process systemd started. Its `/api/system` says `update: self`. A package-managed program that updates itself is uncommon on Linux, where the textbook is an APT repository; that needs root to update from the map, and the network-facing service stays unprivileged, so the service owns a copy of the program the package seeds, as Syncthing does. + +**Rolling back.** A new program stays pending, marked by `MoonLight.pending`, until it has run a minute. A pending program that fails three starts within two minutes gives way to `MoonLight.prev`, through `moonlight-rollback.service`; a confirmed one stays in place through any number of restarts. An `apt install` is the administrator's choice: it puts the package's program in place, older or newer, and drops `MoonLight.prev`. + +**Running it by hand**, to see its log in a terminal: `sudo systemctl stop moonlight`, then `/usr/lib/moonlight/MoonLight`, which keeps its settings in your own `~/.local/share/MoonLight/`. That copy sits in a folder only root writes, so it is updated by hand; the tarball unpacked in a folder of your own updates itself, restarting in its terminal window. + ## Docker **Use `--network host`.** The web UI and unicast fixture output work through ordinary port mapping (`-p 8080:8080`). mDNS discovery and Art-Net's broadcast mode are multicast, and neither crosses a bridge network. Host networking is what lets a container find boards and be found by them. Verified on a NanoPi R28S driving a ColorLight card. @@ -23,6 +42,7 @@ The facts follow; the steps are MoonWalk's. [Install MoonLight](../moonwalk.md#i | **Config** | `/data/MoonLight/.config/` in the volume (`XDG_DATA_HOME=/data`) | | **Identity** | `/data/MoonLight/.config/identity`, written on the first run | | **Logs** | standard output, so `docker logs` | +| **Updates** | the host's, by Watchtower or Podman's auto-update; the Devices map says so rather than offering an update | | **Output** | Art-Net UDP 6454, DDP 4048 (set by `ddp_port`), E1.31 5568, all outbound | | **Capabilities** | none: it binds its port as an ordinary process | | **Audio input** | a PulseAudio server: [Set up audio input](../moonwalk.md#audio-input) | @@ -34,8 +54,8 @@ The facts follow; the steps are MoonWalk's. [Install MoonLight](../moonwalk.md#i | | Package (`.deb`) | Docker | |---|---|---| -| Updates | download the new `.deb` | `docker pull` and recreate | -| Survives a reboot | needs a systemd unit | `--restart unless-stopped` | +| Updates | from the Devices map, by itself | `docker pull` and recreate, or Watchtower | +| Survives a reboot | the service it installs | `--restart unless-stopped` | | Runs on | Debian family, glibc 2.35+ | any Linux with Docker | | Disk | the binary plus libraries the system already has | an image carrying its own libraries | | Isolation | none: an ordinary program | its own filesystem and process space | diff --git a/docs/reference/metrics/code.json b/docs/reference/metrics/code.json index da02483a1..c906c79f4 100644 --- a/docs/reference/metrics/code.json +++ b/docs/reference/metrics/code.json @@ -3,12 +3,12 @@ "(duplicated lines)": 3186, "(total)": 1607, "blocking call on the render path": 197, - "complex function": 286, + "complex function": 284, "deeply nested": 102, "double in firmware code": 2, "duplicated block": 710, "large file": 24, - "large stack frame": 31, + "large stack frame": 33, "long function": 46, "long parameter list": 33, "module type in the UI": 81, diff --git a/docs/reference/metrics/docgen.json b/docs/reference/metrics/docgen.json index 4a30cea4b..275dc789e 100644 --- a/docs/reference/metrics/docgen.json +++ b/docs/reference/metrics/docgen.json @@ -1,9 +1,9 @@ { - "(total)": 1757, - "appendix sections nothing refers to": 146, + "(total)": 1747, + "appendix sections nothing refers to": 143, "member deep dives": 35, "multi-line comment blocks": 276, "over-wide comment lines": 626, "over-wide doc lines": 33, - "split paragraphs": 641 + "split paragraphs": 634 } diff --git a/docs/reference/metrics/prose.json b/docs/reference/metrics/prose.json index b1c529f0c..6b319bc55 100644 --- a/docs/reference/metrics/prose.json +++ b/docs/reference/metrics/prose.json @@ -1,14 +1,14 @@ { - "(total)": 6901, + "(total)": 6885, "MoonLight.EmDash": 989, "MoonLight.LinkText": 7, "MoonLight.NegatedHeading": 5, - "MoonLight.Negation": 5055, + "MoonLight.Negation": 5042, "MoonLight.SectionLength": 31, "MoonLight.SelfReference": 14, "MoonLight.Spelling": 39, - "MoonLight.Temporal": 171, + "MoonLight.Temporal": 170, "MoonLight.TonePadding": 1, "MoonLight.VagueClose": 25, - "MoonLight.Weasel": 564 + "MoonLight.Weasel": 562 } diff --git a/docs/reference/metrics/repo-health.json b/docs/reference/metrics/repo-health.json index 4888425d5..ceb27475a 100644 --- a/docs/reference/metrics/repo-health.json +++ b/docs/reference/metrics/repo-health.json @@ -1,7 +1,7 @@ { - "commit": "cd6d48389", + "commit": "e6b2e9945", "flash": { - "esp32s3-n16r8": 2493024, + "esp32s3-n16r8": 2493520, "esp32": 2283888, "esp32p4rev1-eth": 2617552, "esp32p4rev1-eth-wifi": 2876672, @@ -14,7 +14,7 @@ "esp32p4rev3-eth": 2615456, "esp32s3-zero": 2125456, "esp32-pico": 2432432, - "desktop-macos": 2583472, + "desktop-macos": 2605824, "desktop-windows": 1925120, "esp32c3": 2405632, "desktop-linux": 2548000, @@ -28,14 +28,14 @@ "esp32s31": "2026-11-11", "esp32": "2026-11-12", "esp32-pico": "2026-11-10", - "esp32s3-n16r8": "2026-11-12", + "esp32s3-n16r8": "2026-11-13", "esp32s3-n8r8": "2026-10-30", "esp32s3-zero": "2026-10-30", "esp32-16mb": "2026-10-30", - "esp32p4rev1-eth-wifi": "2026-11-12", + "esp32p4rev1-eth-wifi": "2026-11-13", "esp32-eth": "2026-11-11", "esp32p4rev3-eth": "2026-11-11", - "desktop-macos": "2026-11-12", + "desktop-macos": "2026-11-13", "desktop-windows": "2026-09-30", "esp32c3": "2026-11-12", "desktop-linux": "2026-10-12", @@ -841,16 +841,16 @@ } }, "loc": { - "core": 37828, + "core": 37899, "light": 35148, - "platform": 25337, - "ui": 14578, - "test": 86476, - "moondeck": 35525 + "platform": 25805, + "ui": 14557, + "test": 86798, + "moondeck": 35531 }, "comments": { "core": { - "lines": 7430, + "lines": 7445, "ratio": 0.218 }, "light": { @@ -858,37 +858,37 @@ "ratio": 0.233 }, "platform": { - "lines": 4482, - "ratio": 0.198 + "lines": 4518, + "ratio": 0.196 }, "ui": { - "lines": 3640, + "lines": 3623, "ratio": 0.264 }, "test": { - "lines": 7950, + "lines": 7971, "ratio": 0.106 }, "moondeck": { - "lines": 4512, + "lines": 4509, "ratio": 0.146 } }, "tests": { - "cases": 3250, + "cases": 3260, "scenarios": 37 }, "docs": { "md_files": 122, - "md_lines": 19691, + "md_lines": 19773, "plans_files": 14, "backlog_lines": 2835, "lessons_lines": 455, "claude_md_lines": 139 }, "complexity": { - "functions": 6276, - "over_threshold": 335, + "functions": 6309, + "over_threshold": 333, "worst_ccn": 128 } } diff --git a/docs/reference/metrics/repo-health.md b/docs/reference/metrics/repo-health.md index 46889c023..571937dd4 100644 --- a/docs/reference/metrics/repo-health.md +++ b/docs/reference/metrics/repo-health.md @@ -1,6 +1,6 @@ # Repo health -Measured at `cd6d48389`. Generated by [`moondeck/check/repo_health.py`](../../../moondeck/check/repo_health.py) on every KPI-gate run. **Do not edit by hand.** +Measured at `e6b2e9945`. Generated by [`moondeck/check/repo_health.py`](../../../moondeck/check/repo_health.py) on every KPI-gate run. **Do not edit by hand.** Current state only; the trend is this file's git history (`git log -p docs/reference/metrics/repo-health.md`). Nothing here fails a build: the numbers make growth visible, the judgment stays human. @@ -8,7 +8,7 @@ Current state only; the trend is this file's git history (`git log -p docs/refer | Metric | v7.0.0 | Now | Change | |---|---:|---:|---:| -| flash desktop-macos | 2,221 KB | 2,523 KB | +301.6 KB (+13.6%) | +| flash desktop-macos | 2,221 KB | 2,545 KB | +323.5 KB (+14.6%) | | flash esp32 | 2,095 KB | 2,230 KB | +135.5 KB (+6.5%) | | flash esp32-16mb | 2,343 KB | 2,210 KB | -133.1 KB (-5.7%) | | flash esp32-eth | 1,642 KB | 1,827 KB | +185.6 KB (+11.3%) | @@ -19,7 +19,7 @@ Current state only; the trend is this file's git history (`git log -p docs/refer | flash esp32p4rev1-eth-wifi | 2,606 KB | 2,809 KB | +202.8 KB (+7.8%) | | flash esp32p4rev3-eth | 2,357 KB | 2,554 KB | +196.7 KB (+8.3%) | | flash esp32p4rev3-eth-wifi | - | 2,612 KB | new | -| flash esp32s3-n16r8 | 2,300 KB | 2,435 KB | +134.4 KB (+5.8%) | +| flash esp32s3-n16r8 | 2,300 KB | 2,435 KB | +134.9 KB (+5.9%) | | flash esp32s3-n32r16 | 2,322 KB | 2,431 KB | +109.3 KB (+4.7%) | | flash esp32s3-n8r2 | - | 2,194 KB | new | | flash esp32s3-n8r8 | 2,277 KB | 2,301 KB | +23.8 KB (+1.0%) | @@ -29,14 +29,14 @@ Current state only; the trend is this file's git history (`git log -p docs/refer | free heap esp32 | 33,034 KB | 33,034 KB | +0.0 KB (+0.0%) | | tick Effects_pipeline_builds_and_renders (desktop-macos) | 7 µs | 7 µs | +0 (+0.0%) | | tick Layouts_resize_reallocates_live (desktop-macos) | 10 µs | 10 µs | +0 (+0.0%) | -| lines core | 33,397 | 37,828 | +4,431 (+13.3%) | +| lines core | 33,397 | 37,899 | +4,502 (+13.5%) | | lines light | 31,681 | 35,148 | +3,467 (+10.9%) | -| lines platform | 22,993 | 25,337 | +2,344 (+10.2%) | -| lines ui | 13,043 | 14,578 | +1,535 (+11.8%) | -| lines test | 76,140 | 86,476 | +10,336 (+13.6%) | -| lines moondeck | 32,680 | 35,525 | +2,845 (+8.7%) | +| lines platform | 22,993 | 25,805 | +2,812 (+12.2%) | +| lines ui | 13,043 | 14,557 | +1,514 (+11.6%) | +| lines test | 76,140 | 86,798 | +10,658 (+14.0%) | +| lines moondeck | 32,680 | 35,531 | +2,851 (+8.7%) | | code findings | 1,642 | 1,607 | -35 (-2.1%) | -| unit tests | 2,836 | 3,250 | +414 (+14.6%) | +| unit tests | 2,836 | 3,260 | +424 (+15.0%) | | scenarios | 36 | 37 | +1 (+2.8%) | Against the release `v7.0.0`, the fixed point a user compares with; the tables below compare against the last commit. A firmware not rebuilt since `v7.0.0` is left out, and a measure the release did not record yet, such as the code check, joins from the next one. @@ -45,27 +45,27 @@ Against the release `v7.0.0`, the fixed point a user compares with; the tables b | Target | Flash | Capacity | Used | Built | |---|---:|---:|---:|:--:| -| desktop-linux | 2,488 KB | - | - | **STALE 31d** | -| desktop-macos | 2,523 KB (+720 B) ⚠ | - | - | yes | -| desktop-windows | 1,880 KB | - | - | **STALE 43d** | -| esp32 | 2,230 KB | 2,496 KB | 89% | yes | -| esp32-16mb | 2,210 KB | 4,096 KB | 54% | **STALE 13d** | -| esp32-eth | 1,827 KB | 2,496 KB | 73% | carried 1d | -| esp32-pico | 2,375 KB | 3,072 KB | 77% | carried 2d | -| esp32-wrover | 2,145 KB | 2,496 KB | 86% | **STALE 13d** | -| esp32c3 | 2,349 KB | 2,496 KB | 94% | yes | -| esp32p4rev1-eth | 2,556 KB | 4,096 KB | 62% | yes | -| esp32p4rev1-eth-wifi | 2,809 KB (+2 KB) ⚠ | 4,096 KB | 69% | yes | -| esp32p4rev3-eth | 2,554 KB | 4,096 KB | 62% | carried 1d | -| esp32p4rev3-eth-wifi | 2,612 KB | 4,096 KB | 64% | **STALE 13d** | -| esp32s3-n16r8 | 2,435 KB (+240 B) ⚠ | 4,096 KB | 59% | yes | -| esp32s3-n32r16 | 2,431 KB | 4,096 KB | 59% | carried 2d | -| esp32s3-n8r2 | 2,194 KB | 3,072 KB | 71% | **STALE 13d** | -| esp32s3-n8r8 | 2,301 KB | 3,072 KB | 75% | **STALE 13d** | -| esp32s3-zero | 2,076 KB | 2,496 KB | 83% | **STALE 13d** | -| esp32s31 | 2,733 KB | 4,096 KB | 67% | carried 1d | +| desktop-linux | 2,488 KB | - | - | **STALE 32d** | +| desktop-macos | 2,545 KB (+22 KB) ⚠ | - | - | yes | +| desktop-windows | 1,880 KB | - | - | **STALE 44d** | +| esp32 | 2,230 KB | 2,496 KB | 89% | carried 1d | +| esp32-16mb | 2,210 KB | 4,096 KB | 54% | **STALE 14d** | +| esp32-eth | 1,827 KB | 2,496 KB | 73% | carried 2d | +| esp32-pico | 2,375 KB | 3,072 KB | 77% | carried 3d | +| esp32-wrover | 2,145 KB | 2,496 KB | 86% | **STALE 14d** | +| esp32c3 | 2,349 KB | 2,496 KB | 94% | carried 1d | +| esp32p4rev1-eth | 2,556 KB | 4,096 KB | 62% | carried 1d | +| esp32p4rev1-eth-wifi | 2,809 KB | 4,096 KB | 69% | yes | +| esp32p4rev3-eth | 2,554 KB | 4,096 KB | 62% | carried 2d | +| esp32p4rev3-eth-wifi | 2,612 KB | 4,096 KB | 64% | **STALE 14d** | +| esp32s3-n16r8 | 2,435 KB (+496 B) ⚠ | 4,096 KB | 59% | yes | +| esp32s3-n32r16 | 2,431 KB | 4,096 KB | 59% | carried 3d | +| esp32s3-n8r2 | 2,194 KB | 3,072 KB | 71% | **STALE 14d** | +| esp32s3-n8r8 | 2,301 KB | 3,072 KB | 75% | **STALE 14d** | +| esp32s3-zero | 2,076 KB | 2,496 KB | 83% | **STALE 14d** | +| esp32s31 | 2,733 KB | 4,096 KB | 67% | carried 2d | | qemu | 1,351 KB | - | - | carried (age?) | -| teensy41 | 1,259 KB | 3,456 KB | 36% | carried 2d | +| teensy41 | 1,259 KB | 3,456 KB | 36% | carried 3d | `Built: yes` was measured this run. `carried (age?)` was not rebuilt either and predates this record, so its age is unknown: it dates itself on the next build. `carried Nd` was NOT rebuilt and its number is N days old, so an absent delta says nothing about the change. **STALE** marks a carry older than 7 days: the number has gone unchecked long enough that growth will surface later as one jump, blamed on whichever commit happens to rebuild that target. `Used` is against the app slot in the firmware's own partition table. @@ -73,19 +73,13 @@ Against the release `v7.0.0`, the fixed point a user compares with; the tables b | Target | Net | New | Removed | Grown | Shrunk | Other | Since | |---|---:|---:|---:|---:|---:|---:|---| -| esp32 | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | `cd6d48389` | -| esp32c3 | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | `cd6d48389` | -| esp32p4rev1-eth | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | `cd6d48389` | -| esp32p4rev1-eth-wifi | +1.5 KB | +0.4 KB | -0.5 KB | +1.7 KB | -0.2 KB | +0.1 KB | `cd6d48389` | -| esp32s3-n16r8 | +0.2 KB | +0.7 KB | -0.2 KB | +0.2 KB | -0.6 KB | +0.1 KB | `cd6d48389` | +| esp32p4rev1-eth-wifi | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | `e6b2e9945` | +| esp32s3-n16r8 | +0.5 KB | +0.6 KB | -0.9 KB | +0.9 KB | -0.2 KB | -0.0 KB | `e6b2e9945` | Largest moves: -- **esp32**: none -- **esp32c3**: none -- **esp32p4rev1-eth**: none -- **esp32p4rev1-eth-wifi**: `mm::DrumKitEffect::DrumKitEffect()` -396 B, `mm_light_paletteRGB` +262 B, `mm::LayoutPicture::place(mm::Layer const&, short, short) [cl` +230 B, `mm::RtspDriver::serveControl()` +100 B, `mm::Layer::placeByLayout(unsigned long*, unsigned int, mm::C` +94 B -- **esp32s3-n16r8**: `mm::NetworkModule::tickIdle(unsigned long)` +494 B, `mm::platform::(anonymous namespace)::flushImage(mm::platform` -136 B, `mm::NetworkReceiveEffect::countGap(unsigned short, unsigned ` +104 B, `mm::NetworkModule::retryKnown(unsigned long)` -94 B, `mm::NetworkReceiveEffect::runReceiveLoop()` -92 B +- **esp32p4rev1-eth-wifi**: none +- **esp32s3-n16r8**: `mm::NetworkModule::tickIdle(unsigned long)` -494 B, `mm::platform::http_fetch_to_ota(char const*, char*, unsigned` -285 B, `mm::platform::http_fetch_to_ota(char const*, char*, unsigned` +285 B, `_ZZN2mm8moonlive13lightBuiltinsEvENKUlvE_clEv$isra$0` +217 B, `mm::HttpServerModule::readImageUrl(mm::platform::TcpConnecti` +166 B What this commit's flash went to, symbol by symbol against the measurement at `Since`. New is new functionality, removed and shrunk are what a cleanup saved, and other is what no symbol owns, such as merged strings and alignment. A rename reads as removed plus new, which the largest moves expose. @@ -151,26 +145,26 @@ These build a bare pipeline with no optional modules, so a change here is a chan | Area | Lines | Comments | Comment share | |---|---:|---:|---:| -| core | 37,828 (+22) ⚠ | 7,430 | 21.8 % | -| light | 35,148 (+37) ⚠ | 7,277 | 23.3 % (+0.1 %) ⚠ | -| platform | 25,337 (+15) ⚠ | 4,482 | 19.8 % | -| ui | 14,578 | 3,640 | 26.4 % | -| test | 86,476 (+567) ⚠ | 7,950 | 10.6 % | -| moondeck | 35,525 (+2) ⚠ | 4,512 | 14.6 % | +| core | 37,899 (+71) ⚠ | 7,445 | 21.8 % | +| light | 35,148 | 7,277 | 23.3 % | +| platform | 25,805 (+468) ⚠ | 4,518 | 19.6 % (−0.2 %) ✓ | +| ui | 14,557 (−21) ✓ | 3,623 | 26.4 % | +| test | 86,798 (+322) ⚠ | 7,971 | 10.6 % | +| moondeck | 35,531 (+6) ⚠ | 4,509 | 14.6 % | ## Tests | Kind | Count | |---|---:| -| unit cases | 3,250 (+30) ✓ | +| unit cases | 3,260 (+10) ✓ | | scenarios | 37 | ## Complexity | Metric | Value | |---|---:| -| functions | 6,276 (+14) ✓ | -| over threshold | 335 (−2) ✓ | +| functions | 6,309 (+33) ✓ | +| over threshold | 333 (−2) ✓ | | worst CCN | 128 | ## Documentation @@ -178,7 +172,7 @@ These build a bare pipeline with no optional modules, so a change here is a chan | Metric | Value | |---|---:| | markdown files | 122 | -| markdown lines | 19,691 (+3) ⚠ | +| markdown lines | 19,773 (+82) ⚠ | | plan files | 14 | | backlog lines | 2,835 | | lessons lines | 455 | diff --git a/docs/reference/openapi.yaml b/docs/reference/openapi.yaml index c61b252f6..e2ac5fdf6 100644 --- a/docs/reference/openapi.yaml +++ b/docs/reference/openapi.yaml @@ -417,7 +417,7 @@ paths: post: tags: [Firmware] operationId: installFirmwareUrl - summary: Installs firmware the device fetches itself; with a MoonBase it reboots into MoonBase to install. + summary: Installs firmware the device fetches itself and restarts on it; with a MoonBase it reboots into MoonBase to install. requestBody: { $ref: "#/components/requestBodies/Url" } responses: "202": @@ -521,7 +521,7 @@ paths: schema: type: object properties: - url: { type: string, description: "The image to install; omitted, the release's matching `shared-c6-slave-.bin`." } + url: { type: string, maxLength: 255, description: "The image to install; omitted, the release's matching `shared-c6-slave-.bin`." } responses: "202": { $ref: "#/components/responses/Ok" } "400": { $ref: "#/components/responses/Error" } @@ -699,7 +699,8 @@ components: type: object required: [url] properties: - url: { type: string, pattern: "^https?://", description: The image to fetch. } + url: { type: string, pattern: "^https?://", maxLength: 255, description: The image to fetch. } + sha256: { type: string, pattern: "^[0-9a-fA-F]{64}$", description: "The image's SHA-256, which a desktop checks before it installs its new program; a board's image carries its own." } Image: required: true description: The binary image, sent with a Content-Length. @@ -785,7 +786,11 @@ components: properties: version: { type: string, description: The firmware version. } build: { type: string, description: The build id. } - firmware: { type: string, description: "The firmware variant, unknown on a desktop." } + firmware: { type: string, description: "The firmware variant, which names its image firmware--v.bin: a board's variant, or a desktop's desktop--." } + update: + type: string + enum: [self, host, manual] + description: "How it takes a new version: it installs one itself (a board, or a desktop whose program sits in a folder it can write), its host replaces it (a container), or someone does by hand (a program in a folder it cannot write)." mac: { type: string, description: "The MAC address, the device identity." } board: { type: string, description: "The board model tooling set, empty until set." } chip: { type: string } diff --git a/docs/reference/testing.md b/docs/reference/testing.md index 99263c3bc..6cd1e6626 100644 --- a/docs/reference/testing.md +++ b/docs/reference/testing.md @@ -243,7 +243,7 @@ Assert those through a scalar the list drives rather than through the list itsel { "name": "it-survived", "op": "expect_control", "id": "System", "key": "deviceName", "equals": "MM-Bench" } ``` -On a board that is the reboot the endpoint performs. On a desktop the endpoint exits the process and nothing restarts it, so the live runner relaunches the binary with the data directory the exiting instance was using: a restart that came back on different files would prove nothing. In-process the op skips, because the scheduler is the process and exiting it would end the run. +Every device restarts itself on the endpoint: a board reboots, and a desktop starts its program again in place, on the same data directory, so what survives the restart is what was saved. In-process the op skips, because the scheduler is the process and restarting it would end the run. ### Walking first setup, with the computer as the phone diff --git a/docs/work/Plan-20261109 - Devices, groups and the network map.md b/docs/work/Plan-20261109 - Devices, groups and the network map.md index ec77da5c1..d3e7811d5 100644 --- a/docs/work/Plan-20261109 - Devices, groups and the network map.md +++ b/docs/work/Plan-20261109 - Devices, groups and the network map.md @@ -168,13 +168,48 @@ Planned by the product owner (2026-11-10) after step 5 and before step 6, each s - It needs a hardware test: mDNS queries were ended by a pool exhaustion on the ESP32, so the passive listen is a hypothesis until a board runs it. - **Verified by:** a Hue bridge nobody configured on the bench map. -#### Update a Linux box from the map ⏳ +#### Every desktop updates itself through the Firmware module 🧪 (test: an install on this Mac, .204 and a Windows PC from their own Firmware card; the CI build on Linux and Windows) -- One install path that updates itself. The `.deb` stays the install: apt checks the architecture and pulls the libraries, and the package creates a `moonlight` service user and the systemd unit. It ships the binary as a seed copy dpkg tracks, which its post-install script copies to `/var/lib/moonlight/bin`, owned by the service user, when newer than what is there. -- `POST /api/firmware/url` on Linux downloads the release's `MoonLight-linux--v….tar.gz`, checks GitHub's SHA-256 digest, keeps the running binary as `MoonLight.prev`, renames the new one in and exits for systemd's `Restart=`; a binary that fails its first boots (the boot record) puts `.prev` back. No sudo rule and no Docker socket, so the network-facing service never holds root. -- A container is its host's to update, through Watchtower or Podman's auto-update, `:latest` following the rolling prerelease; the app detects it runs in one and the map says so instead of offering the button. -- Costs a MIGRATING entry: the Pi's hand-written unit gives way to the packaged one. Unblocks step 9 for a group that mixes boards and Linux boxes. -- **Verified by:** .204's package updated from the map, and rolled back from a binary made to fail; .156's container named as its host's. +Asked by the product owner (2026-11-12): one way to update any MoonLight device, in two steps, this one and the map using it. A desktop prepares the new version beside the running one and a restart activates it, as a board writes its other slot and reboots into it, through the same routes, the same Firmware card and the same asset naming. The line runs between a native install, which updates itself on every OS, and a container, which its host replaces. Settled with the product owner: the Linux package runs a service enabled on install, rollback by systemd, settings moved by hand, the digest from the browser. + +- **The image is the bare program.** The release publishes each desktop's program as `firmware-desktop---v.bin`, the firmware registry's names (`desktop-linux-arm64`), beside the archives and installers it ships. A desktop build carries that name as its firmware, where it said `unknown`, so every path that builds `firmware--v.bin` reaches a desktop: the Firmware card and its picker, the map, MoonDeck, Home Assistant's update over MQTT. +- **`otaBegin`, `otaWrite`, `otaEnd` on a desktop.** The image goes to `MoonLight.next` beside the running program. `otaEnd` vets it, an executable of this OS and architecture by its header (ELF, Mach-O, PE), as a board vets an image's magic and chip, and checks the SHA-256 where the request names one; it then keeps the running program as `MoonLight.prev` and renames the new one into place (rename(2), atomic; Windows renames the running `.exe` aside, which it allows). The running process keeps its old image. `http_fetch_to_ota` and `/api/firmware/upload` drive it unchanged, libcurl or WinHTTP fetching for the first. The `hasOta` gates become the update mode. +- **The restart activates it.** `platform::reboot()` starts the program again: on macOS and Linux the new program takes over the process in place (`execv`), same terminal window, same process to systemd or a container runtime, so a reboot is never a start their restart counting sees, with every socket marked close-on-exec where it is made; on Windows the new `.exe` starts in the same console and the old one exits. The reboot button shows on every device. +- **The update mode.** `/api/system`'s `update`, and the Firmware card's: `self` for a native install whose program sits in a folder it can write, `host` for a container, `manual` for a program in a folder it cannot write, such as the package's seed under `/usr/lib`. +- **The Linux package runs a service, and the service owns its program.** Bespoke, with its reason: the textbook Linux update is an APT repository, which needs root to update from the map, and the network-facing service never holds root; a self-owned copy beside a package-tracked seed is how Syncthing, a network daemon, replaces its own binary. New for MoonLight and uncommon on Linux, where the package manager is the norm. The `.deb` ships the program as a seed at `/usr/lib/moonlight/MoonLight`, a `moonlight` system user in the `video` group, and `moonlight.service`, which its post-install script enables and starts the way debhelper's snippets do, skipping where no systemd runs. The unit runs as that user with `StateDirectory=moonlight`, settings in `/var/lib/moonlight` (`XDG_DATA_HOME`), and `Restart=always`. The post-install script copies the seed to `/var/lib/moonlight/bin/MoonLight`, owned by the service user, on every install: an `apt install` is the administrator's explicit choice, so it wins over a self-update, older or newer, and drops `.prev`. The menu entry opens `http://localhost:8080/` instead of starting a second copy on the same port. No sudo rule and no Docker socket, so the network-facing service never holds root. +- **Rollback is systemd's, for the service, by boot assessment.** The swap leaves `MoonLight.pending` beside the program, and the new program deletes it once it has stayed up `kBootStableMs`, the minute a board's boot record uses: systemd's Automatic Boot Assessment, for one program. `StartLimitBurst=3` within `StartLimitIntervalSec=120` and `OnFailure=moonlight-rollback.service`: a pending program that fails three starts in two minutes, a crash before `main` included, is replaced by `.prev` and the service started again; a confirmed one is left alone, so restarts of a good program never undo it. Elsewhere `.prev` stays beside the program, to rename back by hand. +- **The Firmware card on a desktop** installs as a board's does, by URL from its picker or by upload, when the update mode is `self`; otherwise it offers the download. The shared install picker tells a desktop's own image (`firmware-desktop-…bin`) from the installer's download of the same name. +- **A hypothesis to test first:** a macOS bundle whose inner binary was replaced still launches, Terminal running the binary itself rather than the sealed bundle. +- **Built (2026-11-13).** The image writer, the swap and the restart in [platform_desktop_update.cpp](../../src/platform/desktop/platform_desktop_update.cpp), the header vetting in [ProgramImage.h](../../src/core/util/ProgramImage.h), the download through the one HTTP client (`httpDownload`), the firmware name from CMake, the sockets closed on exec, `hasOta` gone, the bare programs staged by `package_desktop.py`, the Firmware card installing on `self` and offering the download otherwise, the picker telling a desktop's program from its download, the live scenario leaving a desktop to restart itself. Tests: `unit_platform_update.cpp`, `unit_ProgramImage.cpp`, `ui-update-asset.test.mjs`, `test_package_deb.py`. Verified on this Mac: `firmware: desktop-macos-arm64`, `update: self`, and `/api/reboot` restarting in place, same process, uptime from zero. Linux and Windows compile in CI only so far. +- **Removes:** the archive path, the reboot button hidden on desktops, `unknown` as a desktop's firmware name, the Firmware card's desktop download in place of an install, MoonWalk's hand-written unit steps. **Fixed on the way:** the reboot button hidden for a container (`^docker$` never matched `docker-arm64`). +- **Costs** a MIGRATING entry: the hand-written unit goes, the program moves from `/usr/bin` to `/usr/lib/moonlight`, and settings move from the old user's home to `/var/lib/moonlight` with one copy command. The Dockerfile copies the program from the new seed path. Unblocks step 9 for a group that mixes boards and desktops. +- **Verified by:** .204's package updated from its own Firmware card and rolled back from a program made to fail; this Mac and a Windows PC each updated from their own Firmware card, by URL and by upload, back in the same window on the new version; .156's container saying its host updates it. + +#### The map updates a desktop as it updates a board ⏳ (after the step above) + +- **The same call as for a board or a Teensy:** the node's popup posts `firmware--v.bin` to `/api/firmware/url`, with GitHub's digest, and watches `/api/system` until the device answers on the new version. +- **What only the map adds:** the device's `update` read from `/api/system`; a container's node says its host updates it, a program in a read-only folder that it is updated by hand. +- **Removes** the desktop special cases the map and the update badge carry: the badge opening the release page for every desktop, `desktopAssetPrefix`, the desktop branch of the asset choice. **Fixed on the way:** the update badge read the browser's OS, and always x64 on Linux. +- **Built (2026-11-12), on the archive naming:** the popup offering an update to a `self` device and telling a container its host updates it; the asset choice by the device's own model (`ui-update-asset.test.mjs`). The uniform name reduces it to the board's path. +- **Verified by:** .204, this Mac and a Windows PC each updated from another device's map; .156's container node saying its host updates it. + +#### A container updated from the map, through Watchtower ⏳ + +Asked by the product owner (2026-11-12), after the desktops. + +- **The container stays without the Docker socket**, so the device never holds root on its host. Watchtower holds it, and its HTTP API (`--http-api-update`, a token) starts an update of the containers it watches, the same on Linux, Docker Desktop for macOS and Windows, and a NAS. +- **The map calls it.** A setting for Watchtower's address and token, kept in the browser; a container node (`update: host`) then gets "Update through Watchtower", and the map watches `/api/system` until the container answers on the new version. Without a Watchtower set, the node says its host updates it, as now. About 30 lines of JS, nothing on the device. +- **Podman's auto-update** has no API and stays the host's. +- **Verified by:** .156's container updated from the map through a Watchtower on the NanoPi. + +#### MoonBase and the WiFi coprocessor updated from the map ⏳ + +Asked by the product owner (2026-11-12), after the desktops. + +- **The routes exist.** `POST /api/firmware/moonbase-update-url` takes the release's MoonBase image, and `POST /api/firmware/coprocessor-update-url` with an empty URL takes the image matching the device's own release; the device fetches either itself, as it does the app. +- **What the map needs: when each one is due.** `/api/system` reports a MoonBase that does not match the app (the Firmware card's `moonbaseMismatch`) and an outdated coprocessor (`coprocessorMismatch`), each absent while current. +- **What the map offers.** A node's popup shows "Update MoonBase" and "Update WiFi coprocessor" beside the app's update, only when due, each pressed twice like the app's, posting to its route and watching `/api/system` until the peer reports it current. A board without TLS is sent to its own Firmware card, as for the app. +- **Verified by:** a bench board with an older MoonBase and the P4 with an older coprocessor image, each brought current from another device's map. #### WLED to MoonLight from the map ✅ @@ -361,6 +396,7 @@ Seeing the network: Acting on devices and groups: +- **Signed releases, every device**: a release signature (minisign or Ed25519, the public key compiled in) checked before any install, board or desktop, as Sparkle and apt check theirs; the SHA-256 proves integrity, the signature would prove origin beyond TLS to GitHub. Asked 2026-11-13, deferred by the product owner. - **Hue paired once**, the bridge identity moving from each HueDriver into Devices. - **The command half for foreign devices** (`DevicePlugin::command()`): WLED brightness, Hue, Tasmota. - **Copy a device's settings to the group**, and a backup of the whole group, on the export step 17 builds. diff --git a/moondeck/ci/check_glibc_floor.py b/moondeck/ci/check_glibc_floor.py index 587753e38..1c7775aa4 100644 --- a/moondeck/ci/check_glibc_floor.py +++ b/moondeck/ci/check_glibc_floor.py @@ -27,7 +27,7 @@ from pathlib import Path FLOOR = (2, 35) # docs/reference/installing-on-linux.md -BINARY = "usr/bin/MoonLight" +BINARY = "usr/lib/moonlight/MoonLight" def ar_members(data: bytes): diff --git a/moondeck/ci/linux/moonlight-rollback.service b/moondeck/ci/linux/moonlight-rollback.service new file mode 100644 index 000000000..f8b6f3ff5 --- /dev/null +++ b/moondeck/ci/linux/moonlight-rollback.service @@ -0,0 +1,7 @@ +# Started by moonlight.service's OnFailure=: puts the previous program back when the one just put in place keeps failing. +[Unit] +Description=MoonLight: the previous program back after a failed update + +[Service] +Type=oneshot +ExecStart=/usr/lib/moonlight/rollback diff --git a/moondeck/ci/linux/moonlight.service b/moondeck/ci/linux/moonlight.service new file mode 100644 index 000000000..0c40acf47 --- /dev/null +++ b/moondeck/ci/linux/moonlight.service @@ -0,0 +1,34 @@ +# MoonLight as a service: installed and started by the .deb, run as its own user from a folder that user owns, so it installs its own +# new version (POST /api/firmware/url) and restarts on it in place; Restart= brings it back after a crash. https://moonmodules.org/MoonLight/reference/installing-on-linux/ +[Unit] +Description=MoonLight +Documentation=https://moonmodules.org/MoonLight/reference/installing-on-linux/ +After=network-online.target +Wants=network-online.target +# Three starts in two minutes end in the failed state, which starts the rollback: a new program that keeps failing before it has run a minute gives way to the one before it. +StartLimitIntervalSec=120 +StartLimitBurst=3 +OnFailure=moonlight-rollback.service + +[Service] +User=moonlight +Group=moonlight +StateDirectory=moonlight +WorkingDirectory=/var/lib/moonlight +Environment=XDG_DATA_HOME=/var/lib/moonlight +ExecStart=/var/lib/moonlight/bin/MoonLight +Restart=always +RestartSec=5 +# systemd.exec's standard confinement: the firmware routes are open to the LAN, so the program gets no privilege, one capability and no write +# access beyond its own folder. /dev stays open for HDMI out through /dev/dri, and writable executable memory for the MoonLive compiler. +NoNewPrivileges=yes +# CAP_NET_RAW alone, for the raw Ethernet frames a ColorLight card takes; ambient, so it survives the in-place restart. +AmbientCapabilities=CAP_NET_RAW +CapabilityBoundingSet=CAP_NET_RAW +ProtectSystem=strict +ReadWritePaths=/var/lib/moonlight +ProtectHome=yes +PrivateTmp=yes + +[Install] +WantedBy=multi-user.target diff --git a/moondeck/ci/linux/postinst b/moondeck/ci/linux/postinst new file mode 100755 index 000000000..733baf14e --- /dev/null +++ b/moondeck/ci/linux/postinst @@ -0,0 +1,23 @@ +#!/bin/sh +# The package's install: a service user, the program it runs in a folder that user owns, and the service running. +set -e +bin=/var/lib/moonlight/bin +if [ "$1" = configure ]; then + getent passwd moonlight >/dev/null || useradd --system --user-group --home-dir /var/lib/moonlight --shell /usr/sbin/nologin moonlight + # HDMI out draws through the kernel's display devices, which the video group opens. + if getent group video >/dev/null; then usermod -a -G video moonlight; fi + install -d -o moonlight -g moonlight /var/lib/moonlight "$bin" + # An apt install is the administrator's choice and wins over a self-update, older or newer, so the copy the rollback would put back goes too. + install -o moonlight -g moonlight -m 755 /usr/lib/moonlight/MoonLight "$bin/MoonLight.next" + mv -f "$bin/MoonLight.next" "$bin/MoonLight" + rm -f "$bin/MoonLight.prev" "$bin/MoonLight.pending" + if [ -e /etc/systemd/system/moonlight.service ]; then + echo "moonlight: /etc/systemd/system/moonlight.service overrides the packaged service: remove it and run 'systemctl daemon-reload' (MIGRATING)" >&2 + fi +fi +# Where systemd runs: a container build unpacks the package and runs none of this. +if [ -d /run/systemd/system ]; then + systemctl daemon-reload + systemctl enable moonlight.service + systemctl restart moonlight.service +fi diff --git a/moondeck/ci/linux/postrm b/moondeck/ci/linux/postrm new file mode 100755 index 000000000..47337b28c --- /dev/null +++ b/moondeck/ci/linux/postrm @@ -0,0 +1,9 @@ +#!/bin/sh +# A purge removes what the package made: the program it ran and the settings beside it. +set -e +if [ "$1" = purge ]; then + rm -rf /var/lib/moonlight +fi +if [ -d /run/systemd/system ]; then + systemctl daemon-reload || true +fi diff --git a/moondeck/ci/linux/prerm b/moondeck/ci/linux/prerm new file mode 100755 index 000000000..0079d4b39 --- /dev/null +++ b/moondeck/ci/linux/prerm @@ -0,0 +1,6 @@ +#!/bin/sh +# The service stopped before its program goes; an upgrade restarts it from postinst. +set -e +if [ "$1" = remove ] && [ -d /run/systemd/system ]; then + systemctl disable --now moonlight.service || true +fi diff --git a/moondeck/ci/linux/rollback b/moondeck/ci/linux/rollback new file mode 100755 index 000000000..16282f1d2 --- /dev/null +++ b/moondeck/ci/linux/rollback @@ -0,0 +1,12 @@ +#!/bin/sh +# The previous program back, when a new one keeps failing before it has proved itself. The swap leaves MoonLight.pending beside the new program, +# which deletes it after a minute up (systemd's boot assessment, for one program); a confirmed program that fails later fails for another +# reason, which the previous one would only hide. +set -eu +bin=/var/lib/moonlight/bin +[ -f "$bin/MoonLight.pending" ] && [ -f "$bin/MoonLight.prev" ] || exit 0 +mv -f "$bin/MoonLight.prev" "$bin/MoonLight" +rm -f "$bin/MoonLight.pending" +echo "moonlight: the new program kept failing, the previous one is back" +systemctl reset-failed moonlight.service +systemctl start moonlight.service diff --git a/moondeck/ci/package_desktop.py b/moondeck/ci/package_desktop.py index b96edc177..b6efab987 100644 --- a/moondeck/ci/package_desktop.py +++ b/moondeck/ci/package_desktop.py @@ -4,10 +4,12 @@ Runs under CI on macOS, Windows and Linux runners. The output lands in `dist/`: macOS arm64: dist/MoonLight-macos-arm64-vX.Y.Z.tar.gz + .dmg - Windows x64: dist/MoonLight-windows-x64-vX.Y.Z.zip + Windows x64: dist/MoonLight-windows-x64-vX.Y.Z.zip + -setup.exe Linux x64: dist/MoonLight-linux-x64-vX.Y.Z.tar.gz + dist/moonlight_X.Y.Z_amd64.deb Linux arm64: dist/MoonLight-linux-arm64-vX.Y.Z.tar.gz + dist/moonlight_X.Y.Z_arm64.deb +and beside each the bare program, dist/firmware-desktop---vX.Y.Z.bin, which a running desktop installs over the air. + Each archive carries the executable + a short README.txt with run instructions. The macOS build is ad-hoc signed, which gets it as far as Gatekeeper's "could not @@ -35,6 +37,8 @@ BUILD_DIR_WIN = ROOT / "build" / "windows" BUILD_DIR_LINUX = ROOT / "build" / "linux" DIST_DIR = ROOT / "dist" +# The .deb's service, rollback and maintainer scripts, each a file of its own. +LINUX_DIR = ROOT / "moondeck" / "ci" / "linux" def read_version() -> str: @@ -100,6 +104,16 @@ def configure_and_build_linux(version: str = "") -> Path: return binary +def stage_program(binary: Path, version: str, firmware: str) -> Path: + """The bare program as a firmware image, `firmware--v.bin`, the name a board's image takes and the one the + build names itself by (build_info.cpp), so the Firmware card, the map and Home Assistant install a desktop as they install a board.""" + DIST_DIR.mkdir(exist_ok=True) + out = DIST_DIR / f"firmware-{firmware}-v{version}.bin" + shutil.copy2(binary, out) + print(f"package_desktop: wrote {out}") + return out + + def package_linux(binary: Path, version: str) -> Path: """A .tar.gz and a .deb. No signing: Linux has no Gatekeeper equivalent to satisfy. @@ -122,6 +136,7 @@ def package_linux(binary: Path, version: str) -> Path: finally: readme.unlink(missing_ok=True) print(f"package_desktop: wrote {out}") + stage_program(binary, version, f"desktop-linux-{tar_label}") package_deb(binary, version) return out @@ -151,10 +166,21 @@ def package_deb(binary: Path, version: str) -> Path | None: stage = DIST_DIR / f"deb-{version}" shutil.rmtree(stage, ignore_errors=True) (stage / "DEBIAN").mkdir(parents=True) - (stage / "usr" / "bin").mkdir(parents=True) - shutil.copy2(binary, stage / "usr" / "bin" / "MoonLight") - # A menu entry, so a Linux user launches it the way a macOS user opens the .app: Terminal=true - # gives the same visible, closeable window that shows the log and stops the server when closed. + # The program as a seed the package tracks, which postinst copies into the service user's own folder, where it updates itself; + # the service and its rollback beside it, and the maintainer scripts that install and remove them (moondeck/ci/linux/). + lib = stage / "usr" / "lib" / "moonlight" + lib.mkdir(parents=True) + shutil.copy2(binary, lib / "MoonLight") + shutil.copy2(LINUX_DIR / "rollback", lib / "rollback") + units = stage / "usr" / "lib" / "systemd" / "system" + units.mkdir(parents=True) + for unit in ("moonlight.service", "moonlight-rollback.service"): + shutil.copy2(LINUX_DIR / unit, units / unit) + for script in ("postinst", "prerm", "postrm"): + shutil.copy2(LINUX_DIR / script, stage / "DEBIAN" / script) + (stage / "DEBIAN" / script).chmod(0o755) + (lib / "rollback").chmod(0o755) + # A menu entry opening the running service's page: the service already holds port 8080, so a second copy would find it taken. apps = stage / "usr" / "share" / "applications" apps.mkdir(parents=True) (apps / "moonlight.desktop").write_text( @@ -162,9 +188,9 @@ def package_deb(binary: Path, version: str) -> Path | None: "Type=Application\n" "Name=MoonLight\n" "Comment=Drive large LED installations and DMX fixtures\n" - "Exec=MoonLight\n" + "Exec=xdg-open http://localhost:8080/\n" "Icon=moonlight\n" - "Terminal=true\n" + "Terminal=false\n" "Categories=Graphics;Utility;\n", encoding="utf-8") icons = stage / "usr" / "share" / "icons" / "hicolor" / "256x256" / "apps" icons.mkdir(parents=True) @@ -187,8 +213,8 @@ def package_deb(binary: Path, version: str) -> Path | None: "Maintainer: MoonModules \n" "Description: Drive large LED installations and DMX fixtures\n" " MoonLight renders effects to LED fixtures and DMX, controlled from a\n" - " browser. This is the desktop build; run MoonLight and open\n" - " http://localhost:8080/.\n", encoding="utf-8") + " browser. It installs as the moonlight service, which starts at boot\n" + " and updates itself from its page: open http://localhost:8080/.\n", encoding="utf-8") out = DIST_DIR / f"moonlight_{version}_{deb_arch}.deb" run(["dpkg-deb", "--build", "--root-owner-group", str(stage), str(out)]) shutil.rmtree(stage, ignore_errors=True) @@ -396,6 +422,7 @@ def package_macos(binary: Path, version: str) -> Path: finally: readme.unlink(missing_ok=True) print(f"package_desktop: wrote {out}") + stage_program(binary, version, "desktop-macos-arm64") package_dmg(binary, version) return out @@ -598,6 +625,7 @@ def package_windows(binary: Path, version: str) -> Path: finally: readme.unlink(missing_ok=True) print(f"package_desktop: wrote {out}") + stage_program(binary, version, "desktop-windows-x64") package_windows_installer(binary, version) return out diff --git a/moondeck/ci/release_stage.py b/moondeck/ci/release_stage.py index 2df363b13..715751509 100644 --- a/moondeck/ci/release_stage.py +++ b/moondeck/ci/release_stage.py @@ -60,7 +60,7 @@ && mkdir -p /deps \\ && pulse=$(find /usr/lib -name libpulse.so.0 | head -1) \\ && test -n "$pulse" \\ - && { ldd /rootfs/usr/bin/MoonLight; ldd "$pulse"; echo "libpulse.so.0 => $pulse (0)"; } | awk '/=> \\//{print $3}' | sort -u | grep -vE '/(libc|libm|libstdc\\+\\+|libgcc_s)\\.so' | xargs -I{} cp -L {} /deps/ + && { ldd /rootfs/usr/lib/moonlight/MoonLight; ldd "$pulse"; echo "libpulse.so.0 => $pulse (0)"; } | awk '/=> \\//{print $3}' | sort -u | grep -vE '/(libc|libm|libstdc\\+\\+|libgcc_s)\\.so' | xargs -I{} cp -L {} /deps/ # Both binaries are built on Ubuntu 22.04 and need glibc 2.35 at most, which # check_glibc_floor.py enforces, so debian13 (2.41) runs them. See ../../Dockerfile. # Pinned by digest, not by tag: `cc-debian13` is mutable, so an unpinned base means two @@ -69,7 +69,7 @@ # This digest is the multi-arch INDEX, not one platform's manifest, so the same pin serves # amd64 and arm64; buildx resolves the right manifest per platform. FROM gcr.io/distroless/cc-debian13@sha256:9b615fff20e1a4fad29c2b30562580b212c7dd5e2225236735cca0070ed11c78 -COPY --from=fetch /rootfs/usr/bin/MoonLight /usr/bin/MoonLight +COPY --from=fetch /rootfs/usr/lib/moonlight/MoonLight /usr/bin/MoonLight # The libraries the loader resolved, collected in the fetch stage. Without them the # container dies at startup on "libcurl.so.4: cannot open shared object file", which the # run before publishing now catches. @@ -92,10 +92,11 @@ def emit(key: str, value: str) -> None: def resolve_tag(input_tag: str, ref: str, ref_name: str) -> str: - """The release channel tag: the dispatch input wins, a push to main is `latest`, any other ref is its own name.""" + """The release channel tag: the dispatch input wins, a push to main is `latest`, a pull request builds as the next `latest` + (a -dev version, as the merge would ship it), any other ref is its own name.""" if input_tag: return input_tag - return "latest" if ref == "refs/heads/main" else ref_name + return "latest" if ref == "refs/heads/main" or ref.startswith("refs/pull/") else ref_name def shipping(registry: dict, keep) -> list[str]: diff --git a/moondeck/moondeck_ui/app.js b/moondeck/moondeck_ui/app.js index 1d98e90fc..705116345 100644 --- a/moondeck/moondeck_ui/app.js +++ b/moondeck/moondeck_ui/app.js @@ -1396,10 +1396,12 @@ function renderDevices() { const otaBtn = document.createElement("button"); otaBtn.className = "device-inject"; // same compact style as inject const otaFw = device.firmware && device.firmware !== "unknown" ? device.firmware : ""; + // A desktop's local build is no board image under build/esp32-/: it updates from its own Firmware card, from a release. + const isDesktop = otaFw.startsWith("desktop-"); otaBtn.textContent = "OTA"; - otaBtn.disabled = !otaFw; - otaBtn.title = otaFw - ? `Flash the local ${otaFw} build to this device over WiFi (no USB)` + otaBtn.disabled = !otaFw || isDesktop; + otaBtn.title = isDesktop ? "A desktop updates from its own Firmware card" + : otaFw ? `Flash the local ${otaFw} build to this device over WiFi (no USB)` : "OTA needs the device's firmware known (discover it first)"; otaBtn.addEventListener("click", async (e) => { e.preventDefault(); diff --git a/moondeck/scenario/run_live_scenario.py b/moondeck/scenario/run_live_scenario.py index 6a3d84065..fafef640d 100644 --- a/moondeck/scenario/run_live_scenario.py +++ b/moondeck/scenario/run_live_scenario.py @@ -14,7 +14,6 @@ import copy import signal import json -import os import re import statistics import subprocess @@ -224,8 +223,8 @@ def _detect_target(state: dict, local: bool = True) -> str: ESP32: read FirmwareUpdateModule.firmware (`esp32`, `esp32-eth`, `esp32-eth-wifi`, `esp32s3-n16r8`, …): set at compile time from MM_FIRMWARE_NAME and - exposed through the `firmware` control. Desktop: same key but reports - `unknown`, so we substitute desktop- using the runtime os name (still + exposed through the `firmware` control. Desktop: its firmware names the release's program + (`desktop-macos-arm64`), and its metrics key is this host's, desktop- (still distinguishes macOS vs Linux vs Windows builds, which can differ in tick noticeably). See docs/explanation/architecture/index.md § Firmware vs board. @@ -240,7 +239,7 @@ def _detect_target(state: dict, local: bool = True) -> str: firmware = c.get("value") elif m.get("type") == "SystemModule" and c.get("name") == "deviceModel": model = c.get("value") - if firmware and firmware != "unknown": + if firmware and firmware != "unknown" and not firmware.startswith("desktop-"): return firmware if not local: return "desktop-" + str(model or "remote").strip().lower().replace(" ", "-") @@ -470,17 +469,9 @@ def _restore_controls(client, prior: dict) -> int: def _reboot_and_wait(client, target: str, timeout_s: float = 60.0) -> str: """Restart the device and wait for it to answer again, so a scenario can prove what survives. - On a board this is the reboot the endpoint performs. On a desktop the same endpoint EXITS the - process and nothing restarts it, so the runner relaunches the binary itself, with the data - directory the exiting instance was using: a restart that came back on different files would - prove nothing about persistence. Returns "" on success, or the reason it did not come back. + Every device restarts itself on the endpoint, a board by rebooting and a desktop by starting its program again, in place, + on the same data directory. Returns "" on success, or the reason it did not come back. """ - data_dir = os.environ.get("MM_DATA_DIR") - # Only a desktop build on THIS machine is relaunched from here. One on another host is brought - # back by whatever runs it there, a container's restart policy or a service manager, and - # launching a local binary for it would start a second, unrelated device on this machine. - is_desktop = target.startswith("desktop-") and client.local - # Uptime before the restart, so the recovered instance can be told from the one still running. # Without it the first answering /api/state is accepted, which on a board is routinely the OLD # instance replying before it goes down: every persistence assertion after that proves nothing. @@ -495,22 +486,6 @@ def _reboot_and_wait(client, target: str, timeout_s: float = 60.0) -> str: except Exception: pass # the device goes away mid-response, which is the expected shape - if is_desktop: - # The same resolver run_desktop.py uses, which picks the NEWEST candidate rather than the - # first that exists: picking by existence served a stale build whose changes read as no-ops, - # and it names the per-host directory, so this works on Linux and Windows too. - sys.path.insert(0, str(ROOT / "moondeck" / "run")) - from run_desktop import _resolve_executable # noqa: E402 - binary = _resolve_executable() - if not binary.exists(): - return f"no desktop binary to relaunch (looked for {binary})" - time.sleep(1.0) # let the old process release the port before the new one binds it - env = dict(os.environ) - if data_dir: - env["MM_DATA_DIR"] = data_dir - subprocess.Popen([str(binary)], cwd=str(ROOT), env=env, - stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL) - # Evidence of a restart, not merely of an answer: either the device went away and came back, or # it answers with an uptime lower than before. One of the two must hold before this reports success. deadline = time.time() + timeout_s diff --git a/moonwalk/02-install.yaml b/moonwalk/02-install.yaml index 6d21da0a5..22f3916e6 100644 --- a/moonwalk/02-install.yaml +++ b/moonwalk/02-install.yaml @@ -191,11 +191,7 @@ processes: curl -fsSL -o moonlight.deb "$url" && sudo apt install -y ./moonlight.deb ``` - `latest` is the rolling build from main; for the newest tagged release, use `releases/latest`. Or unpack the `.tar.gz` and run it in place. To start it on every boot: [Keep MoonLight running on a Linux board](moonwalk.md#run-linux-board). - - id: run-linux - say: Start MoonLight. - given: [{ input: platform, equals: Linux }] - when: { run: { command: "MoonLight" } } + `latest` is the rolling build from main; for the newest tagged release, use `releases/latest`. The package runs MoonLight as a service, updated from the Devices map: [the service](reference/installing-on-linux.md#the-service). Or unpack the `.tar.gz` and run it in place. - id: docker-run section: Run the container say: Run the published image. @@ -228,7 +224,7 @@ processes: detail: | ![MoonLight running: the web interface with a live 3D preview and the Effects panel](assets/tutorials/windows-05-running.png) - The window MoonLight runs in, a console on Windows or your terminal, **is** the application. It shows the log, and closing it stops MoonLight. The line that matters on a first run is `MoonLight is running: http://localhost:8080/`. The address printed below it is the same interface, reachable from your phone or another machine on your network. + On Windows and macOS the window MoonLight runs in, a console or your terminal, **is** the application. It shows the log, and closing it stops MoonLight. A first run prints `MoonLight is running: http://localhost:8080/`. On Linux it runs as a service, its log in `journalctl -u moonlight`. The same interface answers on the machine's address, from your phone too. On a first install you get a default grid and a running effect, enough to confirm everything works. [How MoonLight works](explanation/how-moonlight-works.md) explains the Layouts, Effects and Drivers down the left-hand side. @@ -414,35 +410,13 @@ processes: - id: install say: Install the package on the board. when: { process: install } - - id: unit - say: Write a systemd unit for MoonLight. - when: { manual: "Save it as /etc/systemd/system/moonlight.service." } - detail: | - ```ini - [Unit] - Description=MoonLight - After=network-online.target - Wants=network-online.target - - [Service] - ExecStart=/usr/bin/MoonLight - Restart=always - RestartSec=5 - User=pi - - [Install] - WantedBy=multi-user.target - ``` - - `Restart=always` covers a crash as well as a reboot. Adjust `User`, and the path for a source build: where you built, such as `/home/pi/MoonLight/build/linux/MoonLight`. For HDMI out, that user is in the `video` group (`sudo usermod -aG video `), which Raspberry Pi OS's first user already is. A container does this with `--restart unless-stopped` instead. - - id: enable - say: Enable the service at boot. - when: { run: { command: "sudo systemctl enable --now moonlight" } } + detail: | + The package installs `moonlight.service`, enables it and starts it. `Restart=always` brings it back after a crash as well as a reboot, and a new version installed from the Devices map starts the same way. A container does this with `--restart unless-stopped` instead. - id: status say: Check the service runs. when: { run: { command: "systemctl status moonlight" } } detail: | - `sudo systemctl stop moonlight` stops MoonLight cleanly, its drivers releasing their outputs. To stop the board: `sudo shutdown now`, since an SD card cut off mid-write can corrupt the filesystem. The service comes back at the next boot. + `journalctl -u moonlight` shows its log. `sudo systemctl stop moonlight` stops MoonLight cleanly, its drivers releasing their outputs. To stop the board: `sudo shutdown now`, since an SD card cut off mid-write can corrupt the filesystem. The service comes back at the next boot. - id: container-overlay-root title: Start the container on an overlay root actor: user diff --git a/src/core/system/FirmwareUpdateModule.h b/src/core/system/FirmwareUpdateModule.h index d44c0c2a5..614b22f2f 100644 --- a/src/core/system/FirmwareUpdateModule.h +++ b/src/core/system/FirmwareUpdateModule.h @@ -126,6 +126,9 @@ class FirmwareUpdateModule : public MoonModule { // On either tab, for the update signals: the app's version, and MoonBase's only while it does not match, so this module is the one place that decides. controls_.addReadOnly("appVersion", versionStr_, sizeof(versionStr_)); controls_.setHidden(controls_.count() - 1, true); + std::snprintf(updateStr_, sizeof(updateStr_), "%s", platform::updateModeName(platform::updateMode())); + controls_.addReadOnly("update", updateStr_, sizeof(updateStr_)); + controls_.setHidden(controls_.count() - 1, true); if (moonbaseOutdated_) { controls_.addReadOnly("moonbaseMismatch", moonbaseStr_, sizeof(moonbaseStr_)); controls_.setHidden(controls_.count() - 1, true); @@ -267,6 +270,8 @@ class FirmwareUpdateModule : public MoonModule { uint32_t totalSnap_ = 0; ///< the total the progress control was bound at /// The installed version. char versionStr_[32] = {}; + /// How this device takes a new version: self, host or manual. + char updateStr_[8] = {}; /// The git id, then the build date. char buildStr_[48] = {}; /// The build variant; on the MoonBase tab, its chip. diff --git a/src/core/system/HttpServerModule.cpp b/src/core/system/HttpServerModule.cpp index 9e14744c3..c06509ac1 100644 --- a/src/core/system/HttpServerModule.cpp +++ b/src/core/system/HttpServerModule.cpp @@ -760,6 +760,12 @@ void HttpServerModule::sendResponse(platform::TcpConnection& conn, int status, c conn.write(reinterpret_cast(body), bodyLen); } +void HttpServerModule::sendError(platform::TcpConnection& conn, int status, const char* why) { + char msg[96]; // the longest reason, the OTA status, is 63 characters + std::snprintf(msg, sizeof(msg), "{\"error\":\"%s\"}", why); + sendResponse(conn, status, "application/json", msg); +} + // The /api/file endpoints: a file body is not a control value; parseFilePath vets the `path=` query (rejects "..", roots at the mount), the one traversal guard every filesystem entry shares. // Copy the `path=` query value into `out` (decoding %XX and '+'), rooted at the mount; false on a missing path or a ".." traversal. @@ -1060,14 +1066,12 @@ static const char* imageUnavailable(platform::OtaTarget target) { // A firmware image moves through a transfer slot as a file upload does, its bytes pushed into the platform's writer as they arrive, so no frame waits on the sender. void HttpServerModule::handleFirmwareUpload(platform::TcpConnection& conn, platform::OtaTarget target, const char* initialBody, size_t initialLen, size_t contentLen) { - if constexpr (!platform::hasOta) { - sendResponse(conn, 501, "application/json", "{\"error\":\"OTA not supported on this platform\"}"); + if (platform::updateMode() != platform::UpdateMode::Self) { + sendResponse(conn, 501, "application/json", "{\"error\":\"this device is updated by its host or by hand\"}"); return; } if (const char* why = imageUnavailable(target)) { - char msg[96]; - std::snprintf(msg, sizeof(msg), "{\"error\":\"%s\"}", why); - sendResponse(conn, 409, "application/json", msg); + sendError(conn, 409, why); return; } // One OTA at a time, as handleFirmwareUrl guards: both write g_ota* state. @@ -1233,32 +1237,20 @@ static void matchingCoprocessorUrl(char* url, size_t len) { } void HttpServerModule::handleImageUrl(platform::TcpConnection& conn, const char* body, platform::OtaTarget target) { - if constexpr (!platform::hasOta) { - sendResponse(conn, 501, "application/json", "{\"error\":\"OTA not supported on this platform\"}"); + if (platform::updateMode() != platform::UpdateMode::Self) { + sendResponse(conn, 501, "application/json", "{\"error\":\"this device is updated by its host or by hand\"}"); return; } if (const char* why = imageUnavailable(target)) { - char msg[96]; - std::snprintf(msg, sizeof(msg), "{\"error\":\"%s\"}", why); - sendResponse(conn, 409, "application/json", msg); + sendError(conn, 409, why); return; } if (otaInFlight()) { sendResponse(conn, 409, "application/json", "{\"error\":\"ota already in progress\"}"); return; } - char url[512] = {}; - mm::json::parseString(body, "url", url, sizeof(url)); - if (url[0] == 0 && target == platform::OtaTarget::Coprocessor) matchingCoprocessorUrl(url, sizeof(url)); - if (url[0] == 0) { - sendResponse(conn, 400, "application/json", "{\"error\":\"url required\"}"); - return; - } - if (std::strncmp(url, "http://", 7) != 0 && std::strncmp(url, "https://", 8) != 0) { - sendResponse(conn, 400, "application/json", - "{\"error\":\"url must start with http:// or https://\"}"); - return; - } + char url[kMaxImageUrl + 2] = {}; + if (!readImageUrl(conn, body, url, target == platform::OtaTarget::Coprocessor)) return; g_otaBytesRead = 0; g_otaBytesTotal = 0; // The radio chip's install ends in a restart, which drops settings still waiting for their debounce, so they are written now. @@ -1482,9 +1474,9 @@ void HttpServerModule::serveSystem(platform::TcpConnection& conn) { sink.flush(); } -// Who the device is and what it runs, the part of the summary a peer's map shows: its MAC-derived identity, its board, its WiFi signal (0 off WiFi), its light outputs and its effects. +// Who the device is and what it runs, the part of the summary a peer's map shows: its MAC-derived identity, how it takes a new version, its board, its WiFi signal (0 off WiFi), its light outputs and its effects. void HttpServerModule::writeDeviceSummary(JsonSink& sink) { - sink.append("\"mac\":"); + sink.appendf("\"update\":\"%s\",\"mac\":", platform::updateModeName(platform::updateMode())); sink.writeJsonString(platform::macString()); const MoonModule* sys = findModuleByName("System"); sink.append(",\"board\":"); @@ -2086,10 +2078,39 @@ void HttpServerModule::handleReboot(platform::TcpConnection& conn) { platform::reboot(); // noreturn } +// The buffer holds one character past the cap, so a value that fills it was cut short by the reader and is refused rather than fetched truncated. +bool HttpServerModule::readImageUrl(platform::TcpConnection& conn, const char* body, char (&url)[kMaxImageUrl + 2], + bool coprocessorDefault) { + mm::json::parseString(body, "url", url, sizeof(url)); + if (url[0] == 0 && coprocessorDefault) matchingCoprocessorUrl(url, sizeof(url)); + const char* why = url[0] == 0 ? "url required" + : std::strlen(url) > kMaxImageUrl ? "url too long (max 255)" + : !http::isHttpUrl(url) ? "url must start with http:// or https://" + : nullptr; + if (!why) return true; + sendError(conn, 400, why); + return false; +} + +// A MoonBase device runs from its one app slot, so the URL is staged in NVS and the device reboots into MoonBase to install; 202 with {"moonbase":true} marks this flow. +void HttpServerModule::installThroughMoonBase(platform::TcpConnection& conn, const char* url) { + if (!platform::moonbaseStageInstallUrl(url)) { + sendResponse(conn, 500, "application/json", "{\"error\":\"could not stage install url\"}"); + return; + } + std::snprintf(g_otaStatus, sizeof(g_otaStatus), "rebooting"); + FilesystemModule::flushPending(); + sendResponse(conn, 202, "application/json", "{\"ok\":true,\"moonbase\":true}"); + conn.close(); + platform::delayMs(200); + platform::otaBootMoonBase(); + platform::reboot(); // noreturn, boots MoonBase, which installs and reboots back +} + void HttpServerModule::handleFirmwareUrl(platform::TcpConnection& conn, const char* body) { - if constexpr (!platform::hasOta) { + if (platform::updateMode() != platform::UpdateMode::Self) { sendResponse(conn, 501, "application/json", - "{\"error\":\"OTA not supported on this platform\"}"); + "{\"error\":\"this device is updated by its host or by hand\"}"); return; } @@ -2100,17 +2121,8 @@ void HttpServerModule::handleFirmwareUrl(platform::TcpConnection& conn, const ch return; } - char url[512] = {}; - mm::json::parseString(body, "url", url, sizeof(url)); - if (url[0] == 0) { - sendResponse(conn, 400, "application/json", "{\"error\":\"url required\"}"); - return; - } - if (!http::isHttpUrl(url)) { - sendResponse(conn, 400, "application/json", - "{\"error\":\"url must start with http:// or https://\"}"); - return; - } + char url[kMaxImageUrl + 2] = {}; + if (!readImageUrl(conn, body, url, false)) return; // A device without TLS has no firmware fetcher at all, so it says so before starting: the UI then fetches the image itself and uploads it. if (!platform::httpsAvailable()) { sendResponse(conn, 501, "application/json", @@ -2118,26 +2130,9 @@ void HttpServerModule::handleFirmwareUrl(platform::TcpConnection& conn, const ch return; } - // A MoonBase device runs from its one app slot, so stage the URL in NVS and reboot into MoonBase to install; 202 with {"moonbase":true} marks this flow. if (platform::otaHasMoonBase() && !platform::otaRunningMoonBase()) { - // The staged URL crosses into MoonBase through a 256-byte NVS read, so a longer one would fail silently and park the device in MoonBase; reject it here. - if (std::strlen(url) > 255) { - sendResponse(conn, 400, "application/json", - "{\"error\":\"url too long for the MoonBase handoff (max 255)\"}"); - return; - } - if (!platform::moonbaseStageInstallUrl(url)) { - sendResponse(conn, 500, "application/json", - "{\"error\":\"could not stage install url\"}"); - return; - } - std::snprintf(g_otaStatus, sizeof(g_otaStatus), "rebooting"); - FilesystemModule::flushPending(); - sendResponse(conn, 202, "application/json", "{\"ok\":true,\"moonbase\":true}"); - conn.close(); - platform::delayMs(200); - platform::otaBootMoonBase(); - platform::reboot(); // noreturn, boots MoonBase, which installs and reboots back + installThroughMoonBase(conn, url); + return; } // Seed the shared globals so the first WS push shows "starting", not a prior failed attempt's error. @@ -2146,13 +2141,12 @@ void HttpServerModule::handleFirmwareUrl(platform::TcpConnection& conn, const ch g_otaBytesTotal = 0; FilesystemModule::flushPending(); // the install ends in a restart, which drops settings still waiting for their debounce + // The image's digest, which a desktop checks before it installs its new program; a board's image carries its own. + char sha256[65] = {}; + mm::json::parseString(body, "sha256", sha256, sizeof(sha256)); if (!platform::http_fetch_to_ota(url, g_otaStatus, sizeof(g_otaStatus), - &g_otaBytesRead, &g_otaBytesTotal)) { - // The platform may have already written an error string; pass it through. - char err[128]; - std::snprintf(err, sizeof(err), - "{\"error\":\"%s\"}", g_otaStatus[0] ? g_otaStatus : "ota start failed"); - sendResponse(conn, 500, "application/json", err); + &g_otaBytesRead, &g_otaBytesTotal, nullptr, sha256[0] ? sha256 : nullptr)) { + sendError(conn, 500, g_otaStatus[0] ? g_otaStatus : "ota start failed"); // the platform's own reason when it wrote one return; } // 202 Accepted: task running; UI polls FirmwareUpdate.update_status. diff --git a/src/core/system/HttpServerModule.h b/src/core/system/HttpServerModule.h index 461caafd5..fa7c30c48 100644 --- a/src/core/system/HttpServerModule.h +++ b/src/core/system/HttpServerModule.h @@ -806,6 +806,8 @@ class HttpServerModule : public MoonModule, public BinaryBroadcaster { bool anyWsClientOnAccessPoint() const MM_NONBLOCKING; /// Answer a short reply, under 4 KB, written whole: it fits a fresh connection's empty send buffer, and a reboot route needs it out before the restart. A longer one goes through answerJson or answerBlock. void sendResponse(platform::TcpConnection& conn, int status, const char* contentType, const char* body); + /// Answer `{"error":"why"}` with `status`, `why` a reason of at most 63 characters. + void sendError(platform::TcpConnection& conn, int status, const char* why); /// Answer a refused document as `PATCH /api/state` does: the engine's error and where it is, with status 400. void sendRefusal(platform::TcpConnection& conn, const StateDocumentResult& r); /// The child module a delete or replace may act on: found, not top-level, and one the user can remove. @@ -921,6 +923,12 @@ class HttpServerModule : public MoonModule, public BinaryBroadcaster { void handleBootMoonBase(platform::TcpConnection& conn); /// Take a firmware URL and answer at once, the install running on its own task. @xref{two-firmware-install-paths} void handleFirmwareUrl(platform::TcpConnection& conn, const char* body); + /// The longest image URL a device fetches, the MoonBase handoff's limit: its NVS read takes 256 bytes with the terminator. + static constexpr size_t kMaxImageUrl = 255; + /// Read the image URL from `body`, the release's matching radio-chip image when `coprocessorDefault` and none is named; false after answering 400. + bool readImageUrl(platform::TcpConnection& conn, const char* body, char (&url)[kMaxImageUrl + 2], bool coprocessorDefault); + /// Stage `url` for MoonBase and reboot into it, which installs it and boots the app again. + void installThroughMoonBase(platform::TcpConnection& conn, const char* url); /// Install an uploaded image, the application's (`POST /api/firmware/upload`) or MoonBase's (`POST /api/firmware/moonbase-update`), through a transfer slot. void handleFirmwareUpload(platform::TcpConnection& conn, platform::OtaTarget target, const char* initialBody, size_t initialLen, size_t contentLen); diff --git a/src/core/util/ProgramImage.h b/src/core/util/ProgramImage.h new file mode 100644 index 000000000..39f308f91 --- /dev/null +++ b/src/core/util/ProgramImage.h @@ -0,0 +1,74 @@ +// SPDX-License-Identifier: GPL-3.0-or-later +#pragma once + +/// What a desktop program says about itself in its first bytes: which system loads it and which processor runs it. +/// +/// @technical +/// +/// A desktop vets a new version of itself as a board vets an image (`FirmwareImage.h`), by the header its system's loader reads: +/// +/// | System | Format | Magic | Processor field | +/// |--------|--------|-------|-----------------| +/// | Linux | ELF (64-bit) | `7F 'E' 'L' 'F'`, class 2 | `e_machine` at 18: 62 x86-64, 183 AArch64 | +/// | macOS | Mach-O (64-bit) | `CF FA ED FE` | `cputype` at 4: `0x01000007` x86-64, `0x0100000C` arm64 | +/// | Windows | PE | `'M' 'Z'`, then `"PE\0\0"` at the offset at `0x3C` | `Machine` after it: `0x8664` x64, `0xAA64` arm64 | +/// +/// All three store their fields little-endian on these processors. +/// +/// A universal macOS binary is refused, the release shipping one processor per file. + +#include +#include +#include + +namespace mm { + +/// The executable formats a desktop runs. +enum class ProgramSystem : uint8_t { Linux, MacOS, Windows }; +/// The processors a desktop runs on. +enum class ProgramCpu : uint8_t { X64, Arm64 }; + +namespace program_image { +inline uint16_t le16(std::span b, size_t at) { return static_cast(b[at] | (b[at + 1] << 8)); } +inline uint32_t le32(std::span b, size_t at) { + return static_cast(b[at]) | (static_cast(b[at + 1]) << 8) | (static_cast(b[at + 2]) << 16) + | (static_cast(b[at + 3]) << 24); +} + +/// What a header says: whether it is a program at all, the system that loads it, and the processor field it names. +struct Header { + bool program = false; ///< a known executable format at all + ProgramSystem system = ProgramSystem::Linux; ///< the system whose loader reads it + bool x64 = false; ///< built for x86-64 + bool arm64 = false; ///< built for arm64 +}; + +inline Header elf(std::span b) { + if (b.size() < 20 || b[0] != 0x7F || b[1] != 'E' || b[2] != 'L' || b[3] != 'F' || b[4] != 2) return {}; + return {true, ProgramSystem::Linux, le16(b, 18) == 62, le16(b, 18) == 183}; +} + +inline Header machO(std::span b) { + if (b.size() < 8 || le32(b, 0) != 0xFEEDFACFu) return {}; + return {true, ProgramSystem::MacOS, le32(b, 4) == 0x01000007u, le32(b, 4) == 0x0100000Cu}; +} + +inline Header pe(std::span b) { + if (b.size() < 0x40 || b[0] != 'M' || b[1] != 'Z') return {}; + const uint32_t peAt = le32(b, 0x3C); + if (b.size() < static_cast(peAt) + 6 || le32(b, peAt) != 0x00004550u) return {}; + return {true, ProgramSystem::Windows, le16(b, peAt + 4) == 0x8664, le16(b, peAt + 4) == 0xAA64}; +} +} // namespace program_image + +/// Why `head`, a program's first 4 KB, cannot replace the program of `system` on `cpu`, which the platform names, or nullptr when it can. +inline const char* programRejection(std::span head, ProgramSystem system, ProgramCpu cpu) { + program_image::Header h = program_image::elf(head); + if (!h.program) h = program_image::machO(head); + if (!h.program) h = program_image::pe(head); + if (!h.program) return "not a program"; // a 404 page, an archive, a board's image + if (h.system != system) return "a program for another system"; + return (cpu == ProgramCpu::Arm64 ? h.arm64 : h.x64) ? nullptr : "a program for another processor"; +} + +} // namespace mm diff --git a/src/core/util/build_info.h b/src/core/util/build_info.h index 19c6f7c52..f22d4e12f 100644 --- a/src/core/util/build_info.h +++ b/src/core/util/build_info.h @@ -12,7 +12,7 @@ extern const char* const kVersion; extern const char* const kBuildDate; /// The short git hash it was built from, `+` when the tree was dirty, `nogit` without a checkout: what landed on a board. extern const char* const kBuildId; -/// The compiled-binary variant (`MM_FIRMWARE_NAME`), such as `esp32-eth`, or `unknown` for a local desktop build. +/// The compiled-binary variant (`MM_FIRMWARE_NAME`), such as `esp32-eth`, or `desktop-linux-arm64` for a desktop (CMakeLists.txt), `unknown` on a processor no release packages. extern const char* const kFirmwareName; /// The release channel it shipped on (`MM_RELEASE`), such as `latest` or `v1.0.0`, empty for a local build. extern const char* const kRelease; diff --git a/src/core/util/sha256.h b/src/core/util/sha256.h index 3b8d50956..b7b7fe1b2 100644 --- a/src/core/util/sha256.h +++ b/src/core/util/sha256.h @@ -1,31 +1,21 @@ // SPDX-License-Identifier: GPL-3.0-or-later #pragma once -/// SHA-256 as FIPS 180-4 defines it, carried in the tree rather than linked from a library. +/// SHA-256 as FIPS 180-4 defines it, vendored rather than linked: @xref{one-copy-on-every-target|why}; one call over a buffer and a truncated hex: @xref{one-shot-and-truncated|why}. /// /// @technical /// -/// ## Why a copy rather than a library +/// ## One copy on every target /// -/// This is the only cryptographic primitive MoonLight needs, and it needs it on all five targets. -/// OpenSSL does not exist on ESP32, where ESP-IDF ships mbedtls, so linking it would mean an mbedtls path for devices and an OpenSSL path for the desktop. -/// That is two implementations of one function that must agree byte for byte, and a divergence produces identifiers that silently differ between platforms. -/// One vendored file is the smaller thing to own, and the algorithm is frozen. -/// FIPS 180-4 has not changed since 2015 and its test vectors are published, so `unit_sha256.cpp` pins this against them. +/// ESP-IDF ships mbedtls and the desktop OpenSSL, and two implementations of one function must agree byte for byte; one vendored file of a frozen algorithm is less to own. /// -/// ## What it is not +/// `unit_sha256.cpp` pins it against FIPS 180-4's published test vectors. Its users: the MoonStats installation id, and a desktop's new program checked against the release's digest. /// -/// The one use today is the MoonStats installation id. -/// This is no general-purpose crypto layer: no HMAC, no streaming over a socket, no constant-time comparison. -/// Nothing here needs them, and an unused primitive is a maintenance cost with no user. +/// ## One shot and truncated /// -/// ## Why one shot and why truncation is safe +/// Every caller hashes a buffer already in memory, an identifier or a program of a few MB, so `sha256` takes the whole buffer rather than the init, update and final trio. /// -/// `sha256` takes a whole buffer instead of the usual init, update and final trio. -/// Every caller hashes a short buffer already in memory, so the streaming form would be three functions nobody calls with more than one update. -/// -/// `sha256Hex` truncates, which is the standard way to derive a shorter identifier (NIST SP 800-107 section 5.1). -/// At the default of 16 bytes it leaves 128 bits, far past any collision concern for a population of LED controllers. +/// `sha256Hex` truncates, the standard way to derive a shorter identifier (NIST SP 800-107 § 5.1): 16 bytes leave 128 bits, far past any collision concern here. #include #include diff --git a/src/platform/desktop/host_update.h b/src/platform/desktop/host_update.h new file mode 100644 index 000000000..102bae096 --- /dev/null +++ b/src/platform/desktop/host_update.h @@ -0,0 +1,32 @@ +// SPDX-License-Identifier: GPL-3.0-or-later +#pragma once + +/// A desktop installing a new version of itself as a board does, the desktop half of `otaBegin`, `otaWrite`, `otaEnd` and `reboot`, and the seams a test drives it through. + +#include +#include +#include + +namespace mm::platform { + +/// Whether this process runs in a container, whose host replaces the program rather than the program itself. +bool inContainer(); + +/// A download's receiver, handed each chunk as it arrives; false ends the transfer as failed. +using DownloadChunkFn = bool (*)(void* user, const char* data, size_t n); +/// Download `url` into `take` a chunk at a time, following redirects, the answer's size into `total` once known; the HTTP status, or 0. +int httpDownload(const char* url, uint32_t timeoutMs, DownloadChunkFn take, void* user, uint32_t* total); + +/// The running program's file, or the stand-in a test set. +std::filesystem::path programPath(); +/// Stand a scratch file in for the running program, so a test installs over it; an empty path restores the real one. +void setTestProgramPath(const std::filesystem::path& path); + +/// Vet `next` and its `sha256Hex` when named, keep `program` as `.prev` and put `next` in its place, marked `.pending`; false with the reason in `status`. +bool putProgramInPlace(const std::filesystem::path& next, const std::filesystem::path& program, const char* sha256Hex, + char* status, size_t statusLen); + +/// Confirm a pending program once this run has stayed up `kBootStableMs`, which ends the service's rollback window; called once at start. +void confirmProgramWhenStable(); + +} // namespace mm::platform diff --git a/src/platform/desktop/main_desktop.cpp b/src/platform/desktop/main_desktop.cpp index 5d35dc756..96926d0ec 100644 --- a/src/platform/desktop/main_desktop.cpp +++ b/src/platform/desktop/main_desktop.cpp @@ -10,6 +10,7 @@ #include #include "stop_signal.h" // stopOnSigterm +#include "host_update.h" // confirmProgramWhenStable #ifdef MM_MAC_SCREEN #include #include @@ -138,6 +139,7 @@ int main(int argc, char** argv) { // Unbuffer so every line lands in MoonLight.log before a crash. std::setvbuf(stdout, nullptr, _IONBF, 0); std::setvbuf(stderr, nullptr, _IONBF, 0); + mm::platform::confirmProgramWhenStable(); // a newly installed program proves itself by staying up, which ends its rollback window #ifdef _WIN32 // Winsock is initialized by a static RAII guard in platform_desktop.cpp, so it covers test binaries too (they link mm_platform but have their own main()). Nothing to do here. diff --git a/src/platform/desktop/platform_config.h b/src/platform/desktop/platform_config.h index 9d7b9a2b5..c637a2451 100644 --- a/src/platform/desktop/platform_config.h +++ b/src/platform/desktop/platform_config.h @@ -117,9 +117,6 @@ constexpr EthPinConfig ethConfigDefault{ ethNone, 0, -1, -1, -1, -1, false, -1, /// False: no separate WiFi co-processor, so its read-out and control compile out. constexpr bool hasWifiCoprocessor = false; -/// False: no OTA partition to write, so the firmware routes answer a stub instead. -constexpr bool hasOta = false; - /// False: no UART and no WiFi stack, so the provisioning listener is never installed. constexpr bool hasImprov = false; diff --git a/src/platform/desktop/platform_desktop.cpp b/src/platform/desktop/platform_desktop.cpp index 8aca90fb7..c99f6b08f 100644 --- a/src/platform/desktop/platform_desktop.cpp +++ b/src/platform/desktop/platform_desktop.cpp @@ -129,6 +129,7 @@ #include "platform/usbvideo_frames.h" // areaMean: the shrink the device backend also uses #include "core/util/FirmwareImage.h" // identify/moonBaseRejection: shared image vetting #include "core/util/H264Bitstream.h" // where the encoder's frames begin and end +#include "host_update.h" // inContainer: the label and the update mode both read it #include #include @@ -252,9 +253,11 @@ inline void no_sigpipe(int fd) { (void)fd; #endif } +// Closed when the program restarts in place (execv), so the new one binds the same port; every descriptor that outlives a call takes it. +inline void close_on_exec(int fd) { ::fcntl(fd, F_SETFD, FD_CLOEXEC); } inline int open_sock(int domain, int type, int protocol) { const int fd = ::socket(domain, type, protocol); - if (fd >= 0) no_sigpipe(fd); + if (fd >= 0) { no_sigpipe(fd); close_on_exec(fd); } return fd; } inline int make_nonblocking(int fd) { @@ -655,14 +658,25 @@ bool httpsAvailable() MM_NONBLOCKING { namespace { std::atomic testHttpFetch{nullptr}; -/// Where an answer collects as it arrives, refusing what would not fit rather than truncating it. +/// Where an answer collects as it arrives, refusing what would not fit rather than truncating it, or a stream handing each chunk on. struct FetchSink { char* body; size_t cap; size_t len = 0; bool tooBig = false; - /// Take `n` more bytes, false once they would not fit with the terminator. + bool refused = false; ///< the stream turned a chunk down, which ends the transfer as failed + DownloadChunkFn stream = nullptr; ///< a download's receiver, which takes the answer a chunk at a time + void* user = nullptr; + uint32_t* total = nullptr; ///< where a download learns the answer's size, once its headers name it + /// Whether the answer was cut short, by a body too big for the caller or a chunk the stream refused. + bool failed() const { return tooBig || refused; } + /// Take `n` more bytes, false once they would not fit with the terminator or the stream refuses them. bool take(const char* data, size_t n) { + if (stream) { + if (stream(user, data, n)) return true; + refused = true; + return false; + } if (!body || cap == 0) return true; // a caller that keeps no answer if (len + n + 1 > cap) { tooBig = true; return false; } std::memcpy(body + len, data, n); @@ -685,8 +699,22 @@ bool tlsRoom(const char* url) { namespace { #ifdef _WIN32 +/// The answer's body into `sink`, its size into `sink.total` when asked; a download takes a 200's body alone. +void readAnswer(HINTERNET request, DWORD status, FetchSink& sink) { + DWORD contentLength = 0, lengthSize = sizeof(contentLength); + if (sink.total && WinHttpQueryHeaders(request, WINHTTP_QUERY_CONTENT_LENGTH | WINHTTP_QUERY_FLAG_NUMBER, WINHTTP_HEADER_NAME_BY_INDEX, + &contentLength, &lengthSize, WINHTTP_NO_HEADER_INDEX)) + *sink.total = contentLength; + if (sink.stream && status != 200u) return; // DWORD: an unsigned literal keeps MSVC's /W4 quiet + char chunk[1024]; + DWORD got = 0; + while (WinHttpReadData(request, chunk, sizeof(chunk), &got) && got > 0) + if (!sink.take(chunk, got)) break; +} + /// One request over WinHTTP into `sink`; the status, or 0. -int transportFetch(const char* method, const char* url, const char* reqBody, uint32_t timeoutMs, FetchSink& sink) { +int transportFetch(const char* method, const char* url, const char* reqBody, uint32_t timeoutMs, FetchSink& sink, + bool followRedirects = false) { // The system's own transport and certificate store, nothing vendored; the URL is split by WinHTTP's own splitter, never by hand. const int wideLen = MultiByteToWideChar(CP_UTF8, 0, url, -1, nullptr, 0); if (wideLen <= 0) return 0; @@ -722,10 +750,10 @@ int transportFetch(const char* method, const char* url, const char* reqBody, uin WINHTTP_NO_REFERER, WINHTTP_DEFAULT_ACCEPT_TYPES, secure ? WINHTTP_FLAG_SECURE : 0)) { - // No redirect following: a server moving the request somewhere the caller never named is refused. + // A request stays at the address the caller named, a redirect refused; a download follows one, its digest proving the file. DWORD noRedirects = WINHTTP_DISABLE_REDIRECTS; - WinHttpSetOption(request, WINHTTP_OPTION_DISABLE_FEATURE, - &noRedirects, sizeof(noRedirects)); + if (!followRedirects) + WinHttpSetOption(request, WINHTTP_OPTION_DISABLE_FEATURE, &noRedirects, sizeof(noRedirects)); const char* payload = reqBody ? reqBody : ""; const DWORD payloadLen = static_cast(std::strlen(payload)); @@ -741,10 +769,7 @@ int transportFetch(const char* method, const char* url, const char* reqBody, uin WINHTTP_HEADER_NAME_BY_INDEX, &status, &statusSize, WINHTTP_NO_HEADER_INDEX)) { result = static_cast(status); - char chunk[1024]; - DWORD got = 0; - while (WinHttpReadData(request, chunk, sizeof(chunk), &got) && got > 0) - if (!sink.take(chunk, got)) break; + readAnswer(request, status, sink); } } WinHttpCloseHandle(request); @@ -752,11 +777,35 @@ int transportFetch(const char* method, const char* url, const char* reqBody, uin WinHttpCloseHandle(connection); } WinHttpCloseHandle(session); - return sink.tooBig ? 0 : result; + return sink.failed() ? 0 : result; } #elif defined(MM_HAVE_CURL) +/// A download's options, when `download` asks for them: redirects followed, on https only when it starts there, an error page refused, the size to `sink.total`. +void followAsDownload(CURL* curl, const char* url, FetchSink& sink, bool download) { + if (!download) return; + curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 1L); + curl_easy_setopt(curl, CURLOPT_MAXREDIRS, 5L); + curl_easy_setopt(curl, CURLOPT_FAILONERROR, 1L); + // A redirect stays on https, as WinHTTP's default policy keeps it; the string option arrived in libcurl 7.85, the bitmask before it. + if (std::strncmp(url, "https://", 8) == 0) { +#if LIBCURL_VERSION_NUM >= 0x075500 + curl_easy_setopt(curl, CURLOPT_REDIR_PROTOCOLS_STR, "https"); +#else + curl_easy_setopt(curl, CURLOPT_REDIR_PROTOCOLS, static_cast(CURLPROTO_HTTPS)); +#endif + } + if (!sink.total) return; + curl_easy_setopt(curl, CURLOPT_NOPROGRESS, 0L); + curl_easy_setopt(curl, CURLOPT_XFERINFODATA, &sink); + curl_easy_setopt(curl, CURLOPT_XFERINFOFUNCTION, +[](void* user, curl_off_t downloadTotal, curl_off_t, curl_off_t, curl_off_t) -> int { + *static_cast(user)->total = static_cast(downloadTotal); + return 0; + }); +} + /// One request over libcurl into `sink`; the status, or 0. -int transportFetch(const char* method, const char* url, const char* reqBody, uint32_t timeoutMs, FetchSink& sink) { +int transportFetch(const char* method, const char* url, const char* reqBody, uint32_t timeoutMs, FetchSink& sink, + bool followRedirects = false) { // The library's global init is not thread-safe and must run once, so a function-local static does it: the first call initializes and every later one skips. static const bool inited = (curl_global_init(CURL_GLOBAL_DEFAULT) == CURLE_OK); if (!inited) return 0; @@ -772,8 +821,8 @@ int transportFetch(const char* method, const char* url, const char* reqBody, uin // Both verifications are the defaults, set explicitly so a future edit cannot turn them off quietly: without them the transport proves nothing about who answered. curl_easy_setopt(curl, CURLOPT_SSL_VERIFYPEER, 1L); curl_easy_setopt(curl, CURLOPT_SSL_VERIFYHOST, 2L); - // No redirect following: a server moving the request somewhere the caller never named is refused. - curl_easy_setopt(curl, CURLOPT_FOLLOWLOCATION, 0L); + // A request stays at the address the caller named, a redirect refused; a download follows one, its digest proving the file. + followAsDownload(curl, url, sink, followRedirects); curl_easy_setopt(curl, CURLOPT_NOSIGNAL, 1L); // no SIGALRM in a threaded process struct curl_slist* headers = reqBody ? curl_slist_append(nullptr, "Content-Type: application/json") : nullptr; @@ -792,12 +841,26 @@ int transportFetch(const char* method, const char* url, const char* reqBody, uin curl_slist_free_all(headers); curl_easy_cleanup(curl); - return rc == CURLE_OK && !sink.tooBig ? static_cast(status) : 0; + return rc == CURLE_OK && !sink.failed() ? static_cast(status) : 0; } #endif } // namespace +int httpDownload(const char* url, uint32_t timeoutMs, DownloadChunkFn take, void* user, uint32_t* total) { + if (!url || !*url || !take) return 0; +#if defined(_WIN32) || defined(MM_HAVE_CURL) + FetchSink sink{nullptr, 0}; + sink.stream = take; + sink.user = user; + sink.total = total; + return transportFetch("GET", url, nullptr, timeoutMs, sink, true); +#else + (void)timeoutMs; (void)user; (void)total; + return 0; // built without libcurl: a download fails, visibly +#endif +} + int httpFetch(const char* method, const char* url, const char* reqBody, uint32_t timeoutMs, char* body, size_t bodyLen) { if (body && bodyLen) body[0] = '\0'; @@ -814,16 +877,24 @@ int httpFetch(const char* method, const char* url, const char* reqBody, uint32_t #endif } +bool inContainer() { +#if defined(__linux__) + // The marker the runtime itself creates, checked once, since a process cannot move in or out of a container while it runs. + static const bool inside = std::filesystem::exists("/.dockerenv"); + return inside; +#else + return false; +#endif +} + const char* hostPlatform() { // The release packaging's names, so a breakdown lines up with the downloads. A container reports as one, since that changes how it behaves. #if defined(__linux__) - // The marker the runtime itself creates, checked once, since a process cannot move in or out of a container while it runs. - static const bool inContainer = std::filesystem::exists("/.dockerenv"); // The architecture rides along, since a container on a small board and one on a server are different deployments that the label alone made one slice. #if defined(__aarch64__) - if (inContainer) return "docker-arm64"; + if (inContainer()) return "docker-arm64"; #else - if (inContainer) return "docker-x64"; + if (inContainer()) return "docker-x64"; #endif #if defined(__aarch64__) // Every small arm board lands here, and the breakdown exists to count them: naming the other architecture would report each one as a PC. @@ -1694,7 +1765,7 @@ bool ethBindRawInterface(const char* ifName) { if (!ifName || !ifName[0]) return true; // explicit return to capture mode #if defined(__linux__) - const int fd = ::socket(AF_PACKET, SOCK_RAW, htons(ETH_P_ALL)); + const int fd = ::socket(AF_PACKET, SOCK_RAW | SOCK_CLOEXEC, htons(ETH_P_ALL)); if (fd < 0) return false; const unsigned idx = if_nametoindex(ifName); if (idx == 0) { ::close(fd); return false; } @@ -1707,7 +1778,7 @@ bool ethBindRawInterface(const char* ifName) { for (int i = 0; i < 99; i++) { char dev[24]; std::snprintf(dev, sizeof(dev), "/dev/bpf%d", i); - const int fd = ::open(dev, O_RDWR); + const int fd = ::open(dev, O_RDWR | O_CLOEXEC); if (fd < 0) continue; // busy or no permission, try the next ifreq ifr = {}; std::snprintf(ifr.ifr_name, sizeof(ifr.ifr_name), "%s", ifName); @@ -2106,28 +2177,6 @@ bool resolveHost(const char* name, uint8_t ip[4]) { LookupStep resolveHostStep(const char*, uint8_t*) { return LookupStep::Failed; } // Advertising is a device concern, so these are stubs; discovery itself is datagram presence and runs here too, testable over real loopback. -// No update partition here, and the route guards on the capability, so this stub exists for compile coverage only. -bool http_fetch_to_ota(const char* /*url*/, - char* statusBuf, size_t statusBufLen, - uint32_t* bytesReadOut, uint32_t* bytesTotalOut, - const char* /*fallbackUrl*/) { - if (statusBuf && statusBufLen > 0) { - std::snprintf(statusBuf, statusBufLen, "unsupported on desktop"); - } - if (bytesReadOut) *bytesReadOut = 0; - if (bytesTotalOut) *bytesTotalOut = 0; - return false; -} - -// No OTA partition on desktop, call sites guard with `if constexpr (mm::platform::hasOta)`. -bool otaBegin(OtaWrite&, OtaTarget, size_t, char* statusBuf, size_t statusBufLen, uint32_t* bytesOut) { - if (statusBuf && statusBufLen > 0) std::snprintf(statusBuf, statusBufLen, "unsupported on desktop"); - if (bytesOut) *bytesOut = 0; - return false; -} -bool otaWrite(OtaWrite&, const uint8_t*, size_t) { return false; } -bool otaEnd(OtaWrite&, bool) { return false; } - // No partitions on desktop: there is no recovery image and nothing to boot into, unless a host test gives it one to read. static const char* testMoonBaseVersion_ = nullptr; static const char* testMoonBaseBuilt_ = nullptr; @@ -2277,13 +2326,6 @@ bool improvProvisioningInit(const ImprovDeviceInfo& /*info*/, } void improvPoll() MM_NONBLOCKING {} -void reboot() { - // The device is the host process, so exit cleanly and let the supervisor restart it, which matches what the browser's reconnect expects. - std::printf("platform::reboot(): exiting\n"); - std::fflush(stdout); - // Exiting is the reboot here, there being no firmware to restart into; the thread-safety warning describes exactly the abrupt teardown a reboot models. NOLINTNEXTLINE(concurrency-mt-unsafe) - std::exit(0); -} // UdpSocket @@ -2575,6 +2617,7 @@ TcpConnection TcpServer::accept() { int clientFd = ::accept(fd_, nullptr, nullptr); if (clientFd < 0) return TcpConnection(); no_sigpipe(clientFd); + close_on_exec(clientFd); // A non-blocking client socket, since a blocking read's timeout froze the loop when the bytes had not landed on accept. The request is read across a few rapid retries. make_nonblocking(clientFd); #endif diff --git a/src/platform/desktop/platform_desktop_update.cpp b/src/platform/desktop/platform_desktop_update.cpp new file mode 100644 index 000000000..3093f45a1 --- /dev/null +++ b/src/platform/desktop/platform_desktop_update.cpp @@ -0,0 +1,384 @@ +// SPDX-License-Identifier: GPL-3.0-or-later +/// A desktop installing a new version of itself as a board installs an image into its other slot. +/// +/// The image goes to `MoonLight.next` beside the running program; `otaEnd` vets it, keeps the running program as `MoonLight.prev` and renames the new one in. +/// +/// The restart runs it: the pattern of the self-update libraries (go-update, Rust's self_update). A container is its host's to replace. + +#include "host_update.h" +#include "platform/platform.h" +#include "core/util/ProgramImage.h" // the header a new program is vetted by, as a board vets its image's +#include "core/util/sha256.h" // the digest a fetch is checked against + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#ifdef _WIN32 +#include +#else +#include // pthread_sigmask: the new program starts with every signal open +#include // access, execv +#endif +#ifdef __APPLE__ +#include // _NSGetArgc, _NSGetArgv: the command line the restart repeats +#include // _NSGetExecutablePath +#endif + +namespace fs = std::filesystem; + +namespace mm::platform { + +namespace { + +fs::path g_testProgram; // a test's stand-in for the running program, empty for the real one + +/// The system and processor this build runs on, which a new program must be built for. +constexpr ProgramSystem kThisSystem = +#if defined(_WIN32) + ProgramSystem::Windows; +#elif defined(__APPLE__) + ProgramSystem::MacOS; +#else + ProgramSystem::Linux; +#endif +constexpr ProgramCpu kThisCpu = +#if defined(__aarch64__) || defined(_M_ARM64) + ProgramCpu::Arm64; +#else + ProgramCpu::X64; +#endif + +void say(char* status, size_t statusLen, const char* text) { + if (status && statusLen) std::snprintf(status, statusLen, "%s", text); +} + +std::FILE* openFile(const fs::path& path, const char* mode) { +#ifdef _WIN32 + const std::wstring wideMode(mode, mode + std::strlen(mode)); + return ::_wfopen(path.c_str(), wideMode.c_str()); +#else + return std::fopen(path.c_str(), mode); +#endif +} + +fs::path withSuffix(const fs::path& path, const char* suffix) { + fs::path suffixed = path; + suffixed += suffix; + return suffixed; +} + +/// The file's first bytes, where every executable format keeps its header. +std::vector readHead(const fs::path& path) { + std::vector head(4096); + std::FILE* file = openFile(path, "rb"); + head.resize(file ? std::fread(head.data(), 1, head.size(), file) : 0); + if (file) std::fclose(file); + return head; +} + +/// Whether the file's SHA-256 is the one the release names, compared case-insensitively. +bool digestMatches(const fs::path& path, const char* expectedHex) { + if (std::strlen(expectedHex) != kSha256DigestSize * 2) return false; + std::error_code ec; + const auto fileBytes = fs::file_size(path, ec); + if (ec) return false; + std::FILE* file = openFile(path, "rb"); + if (!file) return false; + std::vector bytes(static_cast(fileBytes)); + const bool readWhole = std::fread(bytes.data(), 1, bytes.size(), file) == bytes.size(); + std::fclose(file); + if (!readWhole) return false; + char actualHex[kSha256DigestSize * 2 + 1]; + sha256Hex(bytes.data(), bytes.size(), actualHex, kSha256DigestSize); + for (size_t i = 0; i < kSha256DigestSize * 2; i++) + if (actualHex[i] != std::tolower(static_cast(expectedHex[i]))) return false; + return true; +} + +/// Whether this process may write files in `folder`, which is whether it can put a new program there. +bool folderWritable(const fs::path& folder) { +#ifdef _WIN32 + // Windows' access check reads only the read-only flag, so a probe file asks the file system itself. + const fs::path probe = folder / "MoonLight.probe"; + HANDLE h = ::CreateFileW(probe.c_str(), GENERIC_WRITE, 0, nullptr, CREATE_NEW, + FILE_ATTRIBUTE_TEMPORARY | FILE_FLAG_DELETE_ON_CLOSE, nullptr); + if (h == INVALID_HANDLE_VALUE) return false; + ::CloseHandle(h); + return true; +#else + return ::access(folder.c_str(), W_OK) == 0; +#endif +} + +/// Put `next` at `program`, the running one kept as `.prev`; the error to report, or nullptr once swapped. +const char* swapIn(const fs::path& next, const fs::path& program) { + std::error_code ec; + const fs::path prev = withSuffix(program, ".prev"); + fs::remove(prev, ec); +#ifdef _WIN32 + // A running program cannot be overwritten on Windows, but it can be renamed: the old one steps aside and the new one takes its name. + fs::rename(program, prev, ec); + if (ec) return "error: cannot keep the running program as .prev"; + fs::rename(next, program, ec); + if (ec) { + std::error_code restored; + fs::rename(prev, program, restored); // the old program back under its name + } +#else + fs::permissions(next, fs::perms::owner_all | fs::perms::group_read | fs::perms::group_exec | fs::perms::others_read | fs::perms::others_exec, ec); + // The running program keeps a second name, so the rollback has it to put back and the program's own name holds a whole program at every moment. + fs::create_hard_link(program, prev, ec); + if (ec) return "error: cannot keep the running program as .prev"; + fs::rename(next, program, ec); // rename(2), atomic: the name moves from the old program to the new one in one step +#endif + return ec ? "error: cannot put the new program in place" : nullptr; +} + +/// An image on its way to `MoonLight.next`, which `otaEnd` puts in place. +struct ProgramWrite { + fs::path next; + fs::path program; + std::FILE* file = nullptr; + char* status = nullptr; + size_t statusLen = 0; + uint32_t* bytesOut = nullptr; + uint32_t expected = 0; ///< the image's size once known, for the progress line + uint32_t written = 0; + std::string sha256; ///< the digest a fetch names, empty for an upload +}; + +/// Drop an image that will not be installed. +void abandon(OtaWrite& w) { + std::unique_ptr img(static_cast(w.impl)); + w.impl = nullptr; + if (!img) return; + if (img->file) std::fclose(img->file); + std::error_code ec; + fs::remove(img->next, ec); +} + +#ifndef _WIN32 +/// The command line this run started with, to start the next one the same way. +std::vector commandLine() { + std::vector args; +#ifdef __APPLE__ + args.reserve(static_cast(*_NSGetArgc())); + for (int i = 0; i < *_NSGetArgc(); i++) args.emplace_back((*_NSGetArgv())[i]); +#else + std::FILE* file = std::fopen("/proc/self/cmdline", "rb"); + std::string arg; + for (int c; file && (c = std::fgetc(file)) != EOF;) { + if (c) arg += static_cast(c); + else { args.push_back(arg); arg.clear(); } + } + if (file) std::fclose(file); +#endif + return args; +} +#endif + +/// The fetch's own thread: the image downloaded into the writer, put in place, and the program restarted on it. +void fetchTask(const std::string& url, const std::string& fallbackUrl, const std::string& sha256Hex, char* status, size_t statusLen, + uint32_t* bytesRead, uint32_t* bytesTotal) { + for (const std::string& from : {url, fallbackUrl}) { + if (from.empty()) continue; + OtaWrite w; + if (!otaBegin(w, OtaTarget::App, 0, status, statusLen, bytesRead)) return; + static_cast(w.impl)->sha256 = sha256Hex; + const int httpStatus = httpDownload(from.c_str(), 10 * 60 * 1000, + [](void* user, const char* data, size_t n) { + return otaWrite(*static_cast(user), reinterpret_cast(data), n); + }, + &w, bytesTotal); + if (httpStatus != 200) { + if (w.impl) abandon(w); + say(status, statusLen, "error: the download failed"); + continue; + } + if (otaEnd(w, true)) { + say(status, statusLen, "rebooting"); + reboot(); + } + return; // refused by its header or digest: the other address serves the same file + } +} + +} // namespace + +fs::path programPath() { + if (!g_testProgram.empty()) return g_testProgram; + std::error_code ec; +#if defined(_WIN32) + std::wstring path(32768, L'\0'); + const DWORD n = ::GetModuleFileNameW(nullptr, path.data(), static_cast(path.size())); + path.resize(n); + return fs::path(path); +#elif defined(__APPLE__) + std::string path(4096, '\0'); + uint32_t size = static_cast(path.size()); + if (_NSGetExecutablePath(path.data(), &size) != 0) return {}; + return fs::weakly_canonical(fs::path(path.c_str()), ec); +#else + return fs::read_symlink("/proc/self/exe", ec); +#endif +} + +void setTestProgramPath(const fs::path& path) { g_testProgram = path; } + +UpdateMode updateMode() { + if (inContainer()) return UpdateMode::Host; + const fs::path program = programPath(); + return !program.empty() && folderWritable(program.parent_path()) ? UpdateMode::Self : UpdateMode::Manual; +} + +bool putProgramInPlace(const fs::path& next, const fs::path& program, const char* sha256Hex, char* status, size_t statusLen) { + char why[96]; + const auto refuse = [&](const char* reason) { + std::error_code ignored; + fs::remove(next, ignored); + say(status, statusLen, reason); + return false; + }; + if (const char* rejection = programRejection(readHead(next), kThisSystem, kThisCpu)) { + std::snprintf(why, sizeof(why), "error: the image is %s", rejection); + return refuse(why); + } + if (sha256Hex && *sha256Hex && !digestMatches(next, sha256Hex)) return refuse("error: the image does not match the release's digest"); + if (const char* failure = swapIn(next, program)) return refuse(failure); + // Pending until the new program has run a minute, which is when the service's rollback stops applying to it. + if (std::FILE* pending = openFile(withSuffix(program, ".pending"), "wb")) std::fclose(pending); + return true; +} + +void confirmProgramWhenStable() { + const fs::path pending = withSuffix(programPath(), ".pending"); + std::error_code ec; + if (!fs::exists(pending, ec)) return; + std::thread([pending] { + std::this_thread::sleep_for(std::chrono::milliseconds(kBootStableMs)); + std::error_code ignored; + fs::remove(pending, ignored); + }).detach(); +} + +bool otaBegin(OtaWrite& w, OtaTarget target, size_t contentLen, char* statusBuf, size_t statusBufLen, uint32_t* bytesOut) { + if (bytesOut) *bytesOut = 0; + if (target != OtaTarget::App) { + say(statusBuf, statusBufLen, "error: a desktop has one program, no MoonBase or radio chip"); + return false; + } + if (updateMode() != UpdateMode::Self) { + say(statusBuf, statusBufLen, "error: this program is updated by its host or by hand"); + return false; + } + auto img = std::unique_ptr(new (std::nothrow) ProgramWrite); + if (!img) { say(statusBuf, statusBufLen, "error: out of memory"); return false; } + img->program = programPath(); + img->next = img->program.parent_path() / "MoonLight.next"; + img->status = statusBuf; + img->statusLen = statusBufLen; + img->bytesOut = bytesOut; + img->expected = static_cast(contentLen); + img->file = openFile(img->next, "wb"); + if (!img->file) { say(statusBuf, statusBufLen, "error: cannot write beside the program"); return false; } + say(statusBuf, statusBufLen, "flashing"); + w.impl = img.release(); + return true; +} + +bool otaWrite(OtaWrite& w, const uint8_t* data, size_t len) { + auto* img = static_cast(w.impl); + if (!img) return false; + if (std::fwrite(data, 1, len, img->file) != len) { + say(img->status, img->statusLen, "error: cannot write the new program"); + abandon(w); + return false; + } + img->written += static_cast(len); + if (img->bytesOut) *img->bytesOut = img->written; + if (img->status && img->statusLen) { + if (img->expected) std::snprintf(img->status, img->statusLen, "flashing: %u of %u bytes", static_cast(img->written), static_cast(img->expected)); + else std::snprintf(img->status, img->statusLen, "flashing: %u bytes", static_cast(img->written)); + } + return true; +} + +bool otaEnd(OtaWrite& w, bool commit) { + auto* img = static_cast(w.impl); + if (!img) return false; + const bool closed = std::fclose(img->file) == 0; + img->file = nullptr; + if (!commit || !closed) { + abandon(w); + return false; + } + const std::unique_ptr owned(img); + w.impl = nullptr; + return putProgramInPlace(owned->next, owned->program, owned->sha256.c_str(), owned->status, owned->statusLen); +} + +bool http_fetch_to_ota(const char* url, char* statusBuf, size_t statusBufLen, uint32_t* bytesReadOut, + uint32_t* bytesTotalOut, const char* fallbackUrl, const char* sha256Hex) { + if (bytesReadOut) *bytesReadOut = 0; + if (bytesTotalOut) *bytesTotalOut = 0; + if (!url || !statusBuf || statusBufLen == 0) return false; + if (updateMode() != UpdateMode::Self) { + say(statusBuf, statusBufLen, "error: this program is updated by its host or by hand"); + return false; + } + say(statusBuf, statusBufLen, "downloading"); + std::thread(fetchTask, std::string(url), std::string(fallbackUrl ? fallbackUrl : ""), std::string(sha256Hex ? sha256Hex : ""), + statusBuf, statusBufLen, bytesReadOut, bytesTotalOut).detach(); + return true; +} + +void reboot() { + std::printf("platform::reboot(): restarting\n"); + std::fflush(stdout); + const fs::path program = programPath(); +#ifdef _WIN32 + // The new program starts in this console and takes the port once this one exits; a virus scan can hold a fresh program a moment, so it is retried. + std::wstring commandLine = ::GetCommandLineW(); + if (commandLine.find(L"--no-browser") == std::wstring::npos) commandLine += L" --no-browser"; + for (int attempt = 0; attempt < 10; attempt++) { + STARTUPINFOW startup{}; + startup.cb = sizeof(startup); + PROCESS_INFORMATION process{}; + if (::CreateProcessW(program.c_str(), commandLine.data(), nullptr, nullptr, FALSE, 0, nullptr, nullptr, &startup, &process)) { + ::CloseHandle(process.hThread); + ::CloseHandle(process.hProcess); + std::exit(0); // NOLINT(concurrency-mt-unsafe): the new program runs on + } + std::this_thread::sleep_for(std::chrono::milliseconds(500)); + } + std::printf("platform::reboot(): could not start %ls (error %lu): start MoonLight again by hand\n", program.c_str(), ::GetLastError()); + std::fflush(stdout); +#else + // The new program takes over this process in place, the one systemd or a container started, so their restart counting sees crashes only. + std::vector args = commandLine(); + if (args.empty()) args.push_back(program.string()); + bool quiet = false; + for (const std::string& arg : args) quiet = quiet || arg == "--no-browser"; + if (!quiet) args.emplace_back("--no-browser"); + std::vector argv; + argv.reserve(args.size() + 1); + for (std::string& arg : args) argv.push_back(arg.data()); + argv.push_back(nullptr); + // The sockets close on exec, the listening one among them (close_on_exec where each is made), so the new program binds the port again. + sigset_t none; + sigemptyset(&none); + ::pthread_sigmask(SIG_SETMASK, &none, nullptr); // a thread's blocked signals would carry over and leave Ctrl+C dead + ::execv(program.c_str(), argv.data()); +#endif + std::exit(0); // NOLINT(concurrency-mt-unsafe): on Windows the new program runs on; elsewhere exec failed, and exiting is what a reboot did before +} + +} // namespace mm::platform diff --git a/src/platform/esp32/platform_config.h b/src/platform/esp32/platform_config.h index 887d09f1a..22fad7ee2 100644 --- a/src/platform/esp32/platform_config.h +++ b/src/platform/esp32/platform_config.h @@ -261,9 +261,6 @@ constexpr EthPinConfig ethConfigDefault = /*miso*/ -1, /*mosi*/ -1, /*sck*/ -1, /*cs*/ -1, /*irq*/ -1 }; #endif // CONFIG_ETH_USE_OPENETH -/// True on every ESP32 build, the partition layout reserving both app slots unconditionally. -constexpr bool hasOta = true; - /// True on every ESP32: the serial RPC channel is always there, so the listener always runs. constexpr bool hasImprov = true; diff --git a/src/platform/esp32/platform_esp32.cpp b/src/platform/esp32/platform_esp32.cpp index d19460f6e..f8141c8d8 100644 --- a/src/platform/esp32/platform_esp32.cpp +++ b/src/platform/esp32/platform_esp32.cpp @@ -359,6 +359,9 @@ const char* hostPlatform() { return ""; } +// A board installs every image it is handed, through its own update partition or MoonBase. +UpdateMode updateMode() { return UpdateMode::Self; } + const char* chipModel() { esp_chip_info_t info; esp_chip_info(&info); diff --git a/src/platform/esp32/platform_esp32_ota.cpp b/src/platform/esp32/platform_esp32_ota.cpp index 200c26549..6e09d5594 100644 --- a/src/platform/esp32/platform_esp32_ota.cpp +++ b/src/platform/esp32/platform_esp32_ota.cpp @@ -291,7 +291,7 @@ void otaTask(void* arg) { bool http_fetch_to_ota(const char* url, char* statusBuf, size_t statusBufLen, uint32_t* bytesReadOut, uint32_t* bytesTotalOut, - const char* fallbackUrl) { + const char* fallbackUrl, const char* /*sha256Hex*/) { // the image closes with its own SHA-256, which the update checks if (!url || !statusBuf || statusBufLen == 0 || !bytesReadOut || !bytesTotalOut) { return false; } diff --git a/src/platform/platform.h b/src/platform/platform.h index 910f20910..d5f1a240f 100644 --- a/src/platform/platform.h +++ b/src/platform/platform.h @@ -4,7 +4,7 @@ #include #include #include // std::nothrow_t, for InternalRamObject's allocation -#include "platform_config.h" // hasOta / hasPsram / …: flags this header's contract refers to +#include "platform_config.h" // hasPsram / …: flags this header's contract refers to #include "nonblocking.h" // MM_NONBLOCKING, which every declaration below carries /// The one interface every module reaches hardware through, so the same source drives an ESP32, a Teensy, a Raspberry Pi and a desktop. @@ -904,11 +904,20 @@ LookupStep resolveHostStep(const char* name, uint8_t ip[4]); /// Store the DHCP hostname the next bring-up advertises; call it before ethInit or wifiStaInit. void setHostname(const char* name); -/// Fetch a firmware image from `url` and flash it to the next OTA partition, returning at once; `fallbackUrl` is tried where the first cannot be opened. +/// How this device takes a new version: it installs one itself, the host replaces it (a container), or someone does by hand (a program started from a terminal or an app). +enum class UpdateMode : uint8_t { Self, Host, Manual }; +/// The mode's name, as `/api/system` reports it. +constexpr const char* updateModeName(UpdateMode m) { + return m == UpdateMode::Self ? "self" : m == UpdateMode::Host ? "host" : "manual"; +} +/// How this device takes a new version: a board installs its own images, a desktop installs itself when its program sits in a folder it may write, a container is its host's. +UpdateMode updateMode(); + +/// Fetch a firmware image from `url`, or `fallbackUrl` where that cannot be opened, and install it, returning at once; a desktop checks `sha256Hex`, a board's image its own. bool http_fetch_to_ota(const char* url, char* statusBuf, size_t statusBufLen, uint32_t* bytesReadOut, uint32_t* bytesTotalOut, - const char* fallbackUrl = nullptr); + const char* fallbackUrl = nullptr, const char* sha256Hex = nullptr); /// An image written as its bytes arrive, which whoever receives it pushes a chunk at a time. struct OtaWrite { void* impl = nullptr; }; diff --git a/src/platform/teensy/platform_config.h b/src/platform/teensy/platform_config.h index 8619a1d6e..9a6320675 100644 --- a/src/platform/teensy/platform_config.h +++ b/src/platform/teensy/platform_config.h @@ -101,9 +101,6 @@ constexpr EthPinConfig ethConfigDefault{ ethW5500, 0, -1, -1, -1, -1, false, 12, /// False: no WiFi co-processor. constexpr bool hasWifiCoprocessor = false; -/// True: an uploaded image is staged above the program and copied over it at the reboot. -constexpr bool hasOta = true; - /// True: the USB serial port takes the installer's state documents (APPLY_OP); there is no WiFi to provision. constexpr bool hasImprov = true; diff --git a/src/platform/teensy/platform_teensy.cpp b/src/platform/teensy/platform_teensy.cpp index ebb07b971..41a4373c9 100644 --- a/src/platform/teensy/platform_teensy.cpp +++ b/src/platform/teensy/platform_teensy.cpp @@ -363,6 +363,8 @@ const char* macString() { const char* chipModel() { return "i.MX RT1062"; } const char* hostPlatform() { return ""; } +// The board installs an image it is handed, uploaded, having no TLS to fetch one. +UpdateMode updateMode() { return UpdateMode::Self; } const char* psramType() { return ""; } // The radio, where a board has one, is native: no companion chip to report or update. const char* coprocessorWifi() { return ""; } diff --git a/src/platform/teensy/platform_teensy_ota.cpp b/src/platform/teensy/platform_teensy_ota.cpp index 8fdd3bc1d..1554482b5 100644 --- a/src/platform/teensy/platform_teensy_ota.cpp +++ b/src/platform/teensy/platform_teensy_ota.cpp @@ -4,20 +4,22 @@ /// /// ## One slot, so the copy waits for the reboot /// -/// An ESP32 writes its update into a second app slot and flips a boot pointer. The Teensy's program flash holds one image, so the upload goes into the erased space above it, where nothing runs. -/// The copy over the program happens in `reboot()`, after the HTTP answer is out. A power loss during the upload leaves the running image untouched. -/// One during the copy, a few seconds, leaves the board to PJRC's bootloader chip and its program button, never a hard brick. -/// Both sides work in 64 KB blocks, the flash's largest erase, and the copy skips every block the new image shares with the running one. +/// An ESP32 writes its update into a second app slot; the Teensy's program flash holds one image, so the upload goes into the erased space above it, where nothing runs. +/// +/// The copy over the program happens in `reboot()`, after the HTTP answer is out, in 64 KB blocks, the flash's largest erase, skipping every block the new image shares with the running one. +/// +/// A power loss during the upload leaves the running image untouched; one during the copy, a few seconds, leaves the board to PJRC's bootloader chip and its program button. /// /// ## Why the copy masks every interrupt source /// -/// The application executes from flash, and the copy erases it. -/// The core's flash routines turn interrupts back on after every command. So the copy disables every interrupt source at the NVIC and stops SysTick instead, and nothing can fire into a half-erased image. -/// It runs from ITCM, as the core's flash routines do. It passes each sector through RAM, since the flash cannot be read while it is being programmed. +/// The application executes from flash, which the copy erases, and the core's flash routines turn interrupts back on after every command. +/// +/// So the copy disables every interrupt source at the NVIC and stops SysTick, runs from ITCM, and passes each sector through RAM, since flash cannot be read while it is programmed. /// /// ## The image mark /// /// Every image carries its firmware name, and an upload must carry the running one's: another program would leave the board off the network until a USB flash. +/// /// The search reads its prefix from the mark itself, so the prefix occurs once per image, and that reference keeps the linker from collecting the mark. #include "platform/platform.h" @@ -184,7 +186,7 @@ bool otaEnd(OtaWrite& w, bool commit) { } // GitHub serves release assets over HTTPS only, which this board cannot speak. -bool http_fetch_to_ota(const char*, char* statusBuf, size_t statusBufLen, uint32_t* bytesReadOut, uint32_t* bytesTotalOut, const char*) { +bool http_fetch_to_ota(const char*, char* statusBuf, size_t statusBufLen, uint32_t* bytesReadOut, uint32_t* bytesTotalOut, const char*, const char*) { if (statusBuf && statusBufLen) std::snprintf(statusBuf, statusBufLen, "error: no TLS on this board, upload the .bin instead"); if (bytesReadOut) *bytesReadOut = 0; if (bytesTotalOut) *bytesTotalOut = 0; diff --git a/src/ui/app.js b/src/ui/app.js index 84df7b46d..7009e1382 100644 --- a/src/ui/app.js +++ b/src/ui/app.js @@ -3,7 +3,7 @@ // component used by both the device UI (here, OTA flash) and the GitHub Pages // installer (first flash via Web Serial). Module loading is deferred by // default; entry-point is the WS init at the bottom: no ordering surprises. -import { installPicker } from "./install-picker.js"; +import { installPicker, DESKTOP_DOWNLOAD } from "./install-picker.js"; import { preview } from "./preview3d.js"; import { markExpandable, panelFit, panelRefit, panelGrips, setupPanel } from "./panel.js"; import { encodeGif } from "./gif.js"; @@ -2399,7 +2399,7 @@ function createCard(mod, depth) { // Hand a URL to the device. Every install route answers 202 and works on its own task, so // the overlay reports progress rather than this response. - const installUrl = async (url) => { + const installUrl = async (url, sha256) => { if (kind === "app" && hasMoonBase) { // The app is written FROM MoonBase, so this reboots twice and moonbaseUpdateFlow // carries it across the handover. @@ -2409,7 +2409,7 @@ function createCard(mod, depth) { const res = await fetch(k.urlRoute, { method: "POST", headers: { "Content-Type": "application/json" }, - body: JSON.stringify(url ? { url } : {}), + body: JSON.stringify(url ? { url, sha256: sha256 || undefined } : {}), }); if (res.status === 501 && kind === "app") { await installViaBrowser(url); return; } if (!res.ok) throw new Error(await errorMessage(res)); @@ -2527,7 +2527,8 @@ function createCard(mod, depth) { // the release asset in each case. Re-deriving the chip from SystemModule here was a // second mechanism for a job the device had already done, with a JS transform // ("ESP32-S3" -> "esp32s3") that merely coincided with the build system's naming. - ownFirmwareKey: moonbaseAssetKeyFrom(ctrlValue("firmware")), + // A device its host or its user updates has its card offer the download in place of an install. + ownFirmwareKey: ctrlValue("update") && ctrlValue("update") !== "self" ? DESKTOP_DOWNLOAD : moonbaseAssetKeyFrom(ctrlValue("firmware")), // Device already knows its deviceModel (SystemModule): picker is for releases + // firmware compatibility only. A board picker here would invite the user to // mis-narrow the firmware list. @@ -2537,9 +2538,7 @@ function createCard(mod, depth) { // them above would separate that button from the two dropdowns it acts on. extrasAfterInstall: true, onInstall: async (_firmware, _manifestUrl, binaryUrl, entry) => { - // A desktop build has no OTA: it updates by downloading a new release and being - // replaced by hand. So hand the browser the file rather than POSTing to a route - // that would 501 here. + // A download, for a device its host or its user updates: the browser takes the file, the route would answer 501. if (entry && entry.isDesktop) { const a = document.createElement("a"); a.href = binaryUrl; @@ -2551,7 +2550,7 @@ function createCard(mod, depth) { return; } if (k.confirm && !await k.confirm()) return; - await installUrl(binaryUrl); + await installUrl(binaryUrl, entry?.sha256); }, }); @@ -5147,20 +5146,26 @@ async function peerFirmware(base) { } } -// The release a device could move to and the image that installs it, by the same checks the -// update badge makes for this device; url is null for a desktop, which installs from the release page. +// The image of release `rel` that installs `version` on a device, its address and the SHA-256 GitHub names for it, firmware--v.bin for +// a board and a desktop alike. Null url for a device its host or its user updates, and for one older than its firmware name. +function updateAsset(rel, dev, version) { + const installs = !!dev.firmware && (!dev.update || dev.update === "self"); + const asset = installs ? (rel?.assets || []).find(a => a.name === `firmware-${dev.firmware}-v${version}.bin`) : null; + return { url: asset?.browser_download_url ?? null, sha256: asset?.digest?.replace(/^sha256:/, "") ?? null }; +} + +// The release a device could move to and the asset that installs it, by the same checks the update badge makes for this device. async function peerUpdate(dev) { - const asset = (rel, name) => (rel?.assets || []).find(a => a.name === name)?.browser_download_url ?? null; const stable = await stableUpdate(dev, false); if (stable) { const rel = await cachedJson(`${RELEASES_API}/latest`, "MoonLight.update.latest.v1", false); - return { label: stable, url: dev.isDesktop ? null : asset(rel, `firmware-${dev.firmware}-${stable}.bin`) }; + return { label: stable, ...updateAsset(rel, dev, stable.replace(/^v/, "")) }; } if ((parse(dev.version)?.prerelease.length || 0) === 0) return null; // a stable device is not nudged toward a -dev build const v = await devUpdate(dev, false); if (!v) return null; const rel = await cachedJson(`${RELEASES_API}/tags/latest`, "MoonLight.update.dev.v1", false); - return { label: `latest (${v})`, url: dev.isDesktop ? null : asset(rel, `firmware-${dev.firmware}-v${v}.bin`) }; + return { label: `latest (${v})`, ...updateAsset(rel, dev, v) }; } // The Ethernet boards of WLED's table the migration carries over, by WLED's `eth.type`, as kWledEthernet in WledNetworks.h maps them. @@ -5575,11 +5580,16 @@ function buildPeerSummary(row) { summary.append(btn, idNote); } } - const isDesktop = !fw.firmware || fw.firmware === "unknown"; - line.textContent = `Firmware: ${fw.version} · ${isDesktop ? "desktop" : fw.firmware}`; - const u = await peerUpdate({ version: fw.version, firmware: isDesktop ? null : fw.firmware, isDesktop }); + line.textContent = `Firmware: ${fw.version} · ${fw.firmware || "unknown"}`; + const update = fw.summary?.update; + const u = await peerUpdate({ version: fw.version, firmware: fw.firmware && fw.firmware !== "unknown" ? fw.firmware : null, update }); if (!u) { note.textContent = "Up to date."; return; } - if (!u.url) { note.textContent = `${u.label} is available: a desktop installs it from the release page.`; return; } + if (!u.url) { + note.textContent = update === "host" + ? `${u.label} is available: the container's host updates it, by Watchtower or Podman's auto-update.` + : `${u.label} is available: this device is updated by hand, from the release page.`; + return; + } note.textContent = `${u.label} is available.`; const btn = document.createElement("button"); btn.className = "graph-popup-update"; @@ -5590,7 +5600,7 @@ function buildPeerSummary(row) { return box; } -// Hand the device the image's address, as its own Firmware card does, and watch it come back on a +// Hand the device the image's address and digest, as its own Firmware card does, and watch it come back on a // new version. A device without TLS answers 501 and installs from its own Firmware card instead. async function startPeerUpdate(base, u, oldVersion, note, btn) { btn.disabled = true; @@ -5598,7 +5608,7 @@ async function startPeerUpdate(base, u, oldVersion, note, btn) { let res = null; try { res = await fetch(`${base}/api/firmware/url`, { - method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ url: u.url }) }); + method: "POST", headers: { "Content-Type": "application/json" }, body: JSON.stringify({ url: u.url, sha256: u.sha256 }) }); } catch (_) { /* a reboot can cut the reply off; the watch below sees the outcome */ } if (res && res.status === 501) { note.textContent = "This device has no TLS: open its page and install from its Firmware card."; @@ -7372,22 +7382,7 @@ function updateStatusBar() { const statsEl = document.getElementById("sys-stats"); if (statsEl) setText(statsEl, headerStats(ctrls, uiMode() >= 1)); - // Hide reboot button on desktop builds: platform::reboot() just exits the process, - // which is not useful from the UI and can be mistaken for a crash. - const chipCtrl = ctrls.find(c => c.name === "chip"); const rebootBtn = document.getElementById("reboot-btn"); - if (rebootBtn) { - // Hidden on a desktop, where `reboot()` EXITS THE PROCESS and nothing restarts it. Keyed on - // `deviceModel`, which the desktop platform seeds with its own name (macos-arm64, - // linux-x64, windows-x64, docker) and a board leaves for tooling to fill from the catalog. - // - // NOT on `chip`: that used to read "desktop" and now reads the real architecture (arm64, - // x64), so the old test silently stopped matching and put a process-killing button on every - // desktop card. - const modelCtrl = ctrls.find(c => c.name === "deviceModel"); - const model = String(modelCtrl?.value ?? ""); - rebootBtn.hidden = /^(macos|linux|windows)-|^docker$/.test(model); - } // bootReason → crashed-state styling on reboot button const reasonCtrl = ctrls.find(c => c.name === "bootReason"); @@ -7487,21 +7482,6 @@ async function cachedJson(url, key, force, ttlMs = UPDATE_TTL_MS) { return p; } -// Which release asset would UPDATE this machine? An ESP32 installs a firmware--.bin -// through the Firmware card; a desktop downloads an archive and replaces its own binary by hand. -// Both still want to be TOLD there is a newer version, which is what the badge is for. -// -// A desktop reports firmware "unknown" (there is no variant to name), which is how we tell them -// apart. Before this, that value was truthy and the check looked for a firmware-unknown-*.bin that -// no release has ever contained, so a desktop user was never told anything. -function desktopAssetPrefix() { - const p = (navigator.platform || "") + " " + (navigator.userAgent || ""); - if (/Win/i.test(p)) return "MoonLight-windows-x64-v"; - if (/Mac/i.test(p)) return "MoonLight-macos-arm64-v"; - if (/Linux|X11/i.test(p)) return "MoonLight-linux-x64-v"; - return null; // an OS we do not package: say nothing -} - // The device's app version, its MoonBase's version, and its firmware variant, whichever tab the Firmware card shows. // The card's rows describe one image at a time, so the app's version comes from its hidden `appVersion`, a MoonBase that does not match from `moonbaseMismatch`, and the variant from the System card, which names it on every tab. function deviceFirmwareInfo() { @@ -7510,10 +7490,11 @@ function deviceFirmwareInfo() { if (!fw) return null; const value = (mod, name) => ((mod && mod.controls) || []).find(c => c.name === name)?.value; const version = value(fw, "appVersion"); - const firmware = value(allModules().find(m => m.type === "SystemModule"), "firmware"); - // "unknown" is what a desktop build reports: no ESP32 variant to name. - const isDesktop = !firmware || firmware === "unknown"; - return version ? { version, firmware: isDesktop ? null : firmware, isDesktop, moonbase: value(fw, "moonbaseMismatch") || null } : null; + const sys = allModules().find(m => m.type === "SystemModule"); + const firmware = value(sys, "firmware"); + // "unknown" is what a desktop build older than its own firmware name reports, with no image to match. + return version ? { version, firmware: firmware && firmware !== "unknown" ? firmware : null, update: value(fw, "update"), + moonbase: value(fw, "moonbaseMismatch") || null } : null; } // The release a version ships in: its tag for a stable version, the moving `latest` for a prerelease. @@ -7523,20 +7504,18 @@ function releaseTagOf(version) { // Light the badge for an available update. `tag` is the release the picker should pre-select // (a vX.Y.Z stable tag, or "latest"); `label` is what the badge shows. -function showUpdateBadge(badge, tag, label, isDesktop) { +function showUpdateBadge(badge, tag, label, byHand) { // The arrow alone, so the header fits a phone; what the update is reads in the tooltip and on // the Firmware card the badge opens. setText, not a raw write: this runs on every state patch, // and rewriting identical text still destroys a selection inside the element. setText(badge, "⬆"); - // A desktop cannot install from the Firmware card: it has no OTA, and updating means - // downloading an archive and replacing the binary. So the badge sends it to the release page - // instead of a card that would offer it nothing. - badge.title = isDesktop + // A device updated by its host or by hand takes its update from the release page, which the badge opens. + badge.title = byHand ? `Update available: ${label}. Open the release page to download` : `Firmware update available: ${label}. Open Firmware to install`; badge.setAttribute("aria-label", badge.title); badge.dataset.tag = tag; - badge.dataset.desktop = isDesktop ? "1" : ""; + badge.dataset.byhand = byHand ? "1" : ""; badge.dataset.moonbase = ""; badge.hidden = false; } @@ -7547,7 +7526,7 @@ function showMoonBaseBadge(badge, dev) { badge.title = `MoonBase ${dev.moonbase} does not match the app ${dev.version}. Open Firmware to install the matching one`; badge.setAttribute("aria-label", badge.title); badge.dataset.tag = releaseTagOf(dev.version); - badge.dataset.desktop = ""; + badge.dataset.byhand = ""; badge.dataset.moonbase = "1"; badge.hidden = false; } @@ -7570,13 +7549,8 @@ async function stableUpdate(dev, force) { const rel = await cachedJson(`${RELEASES_API}/latest`, "MoonLight.update.latest.v1", force); if (!rel || !rel.tag_name) return null; const assetNames = (rel.assets || []).map(a => a.name); - const prefix = dev.isDesktop ? desktopAssetPrefix() : null; - const hasBinary = dev.isDesktop - // A desktop is told about a release that actually ships a build for ITS OS: an archive - // named for the platform, rather than a firmware .bin it could not install anyway. - ? !!prefix && assetNames.some(n => n.startsWith(prefix)) - : !dev.firmware || - assetNames.some(n => n === `firmware-${dev.firmware}-${rel.tag_name}.bin`); + // A board or a desktop alike: told about a release that ships its own image, firmware--.bin. + const hasBinary = !!dev.firmware && assetNames.some(n => n === `firmware-${dev.firmware}-${rel.tag_name}.bin`); return (isNewer(rel.tag_name, dev.version) && hasBinary) ? rel.tag_name : null; } @@ -7588,16 +7562,13 @@ async function stableUpdate(dev, force) { // We also require the matching firmware .bin asset so the badge never points at a build the // device can't install. async function devUpdate(dev, force) { - if (!dev.firmware && !dev.isDesktop) return null; // can't match an asset without the key + if (!dev.firmware) return null; // the firmware key names the asset const rel = await cachedJson(`${RELEASES_API}/tags/latest`, "MoonLight.update.dev.v1", force); const v = rel && rel.name; if (!v) return null; // Assets are versioned, not tagged: the `latest` release ships // firmware--v.bin (release.yml stages PREFIX="firmware-...-v$V"). - const prefix = dev.isDesktop ? desktopAssetPrefix() : null; - const hasBinary = dev.isDesktop - ? !!prefix && (rel.assets || []).some(a => a.name.startsWith(prefix)) - : (rel.assets || []).some(a => a.name === `firmware-${dev.firmware}-v${v}.bin`); + const hasBinary = (rel.assets || []).some(a => a.name === `firmware-${dev.firmware}-v${v}.bin`); return (hasBinary && isNewer(v, dev.version)) ? v : null; } @@ -7610,7 +7581,8 @@ async function checkFirmwareUpdate(force) { if (!dev) { badge.hidden = true; return null; } const stableTag = await stableUpdate(dev, force); - if (stableTag) { showUpdateBadge(badge, stableTag, stableTag, dev.isDesktop); return { tag: stableTag, label: stableTag }; } + const byHand = !!dev.update && dev.update !== "self"; + if (stableTag) { showUpdateBadge(badge, stableTag, stableTag, byHand); return { tag: stableTag, label: stableTag }; } // Only a prerelease (-dev…) build follows the moving latest channel; a stable device is // not nudged toward an unreleased build. @@ -7618,7 +7590,7 @@ async function checkFirmwareUpdate(force) { if (onPrerelease) { const devVer = await devUpdate(dev, force); if (devVer) { - showUpdateBadge(badge, "latest", `latest (${devVer})`, dev.isDesktop); + showUpdateBadge(badge, "latest", `latest (${devVer})`, byHand); return { tag: "latest", label: `latest (${devVer})` }; } } @@ -7633,7 +7605,7 @@ function setupUpdateBadge() { const badge = document.getElementById("fw-update-badge"); if (!badge) return; badge.addEventListener("click", () => { - if (badge.dataset.desktop) { + if (badge.dataset.byhand) { const tag = badge.dataset.tag || "latest"; window.open(`https://github.com/MoonModules/MoonLight/releases/tag/${tag}`, "_blank", "noopener"); diff --git a/src/ui/install-picker.js b/src/ui/install-picker.js index e6da41fe9..f23db8f3f 100644 --- a/src/ui/install-picker.js +++ b/src/ui/install-picker.js @@ -210,8 +210,8 @@ function parseFirmwaresFromAssets(assets, tag, moonbaseOnly = false) { firmwares.set(firmware, entry); } } - // A DESKTOP build ships an archive, not a .bin. The device cannot flash itself over the air, so - // these are offered as a DOWNLOAD instead, and the caller decides what that means. Each platform + // A DESKTOP build also ships archives and installers, offered as a DOWNLOAD for a computer that is + // not running MoonLight yet, or one its host or its user updates; the caller decides what that means. Each platform // lists its asset patterns in the firmware registry (DESKTOP_BUILDS), the friendliest form first: // a .dmg to drag, a setup.exe that installs, a .deb apt installs, over the archive that is there // for scripting. Download takes the first pattern the release has an asset for. @@ -222,7 +222,8 @@ function parseFirmwaresFromAssets(assets, tag, moonbaseOnly = false) { for (const a of assets) if (rx.test(a.name)) found.push({ name: a.name, url: a.browser_download_url }); } if (!found.length) continue; - firmwares.set(b.key, { firmware: b.key, manifestUrl: null, binaryUrl: found[0].url, isDesktop: true, assets: found }); + // Keyed apart from the desktop's own program of the same name, which the .bin loop below adds. + firmwares.set(`download:${b.key}`, { firmware: b.key, manifestUrl: null, binaryUrl: found[0].url, isDesktop: true, assets: found }); } for (const a of assets) { @@ -253,14 +254,15 @@ function parseFirmwaresFromAssets(assets, tag, moonbaseOnly = false) { const firmware = m[1]; const entry = firmwares.get(firmware) || { firmware, manifestUrl: null, binaryUrl: null }; entry.binaryUrl = a.browser_download_url; + entry.sha256 = a.digest?.replace(/^sha256:/, "") || null; // GitHub's digest, which a desktop checks before it installs the image firmwares.set(firmware, entry); } } // An OTA firmware needs BOTH a manifest and a binary: a partial upload (the - // mid-release-publish race) shouldn't appear in the dropdown. A desktop archive has no - // manifest by nature (there is nothing to flash), so it qualifies on its download alone. + // mid-release-publish race) shouldn't appear in the dropdown. A desktop archive and a desktop's + // own program qualify on their file alone, a manifest being what a flash over USB reads. return Array.from(firmwares.values()) - .filter(f => f.binaryUrl && (f.manifestUrl || f.isDesktop || f.isMoonBase)); + .filter(f => f.binaryUrl && (f.manifestUrl || f.isDesktop || f.isMoonBase || isDesktopProgram(f.firmware))); } // Merge a release's published firmwares with locally-staged extras (preview only). @@ -279,6 +281,13 @@ function mergeFirmwares(published, extras) { // 4. Compatibility filter (OTA only) // --------------------------------------------------------------------------- +// The picker's download mode, as its own firmware key: the desktop build for the machine viewing the page, on the web installer's +// "This computer" and on the Firmware card of a desktop its host or its user updates. +export const DESKTOP_DOWNLOAD = "desktop-download"; + +// A desktop's own program, firmware-desktop---v.bin, which a running desktop installs over the air. +const isDesktopProgram = (firmware) => /^desktop-(?!download$)/.test(firmware || ""); + // Bespoke rule for MoonLight's firmware keys: strip the `-eth*` suffix from // both sides; equal identities are mutually OTA-compatible. So `esp32` and // `esp32-eth` can flash each other (same physical ESP32 silicon; the variant @@ -288,10 +297,12 @@ function mergeFirmwares(published, extras) { // only compatible with itself (different chip family AND different partition // table). Web installer passes ownFirmwareKey=null → all candidates compatible. function isCompatible(ownFirmwareKey, candidateFirmwareKey) { + // Download mode: the archive built for the OS the browser is running on, so a Mac is offered the .dmg and not the Windows .zip. + // The archive shares its key with the desktop's own program, which the caller's download filter keeps out of this list. + if (ownFirmwareKey === DESKTOP_DOWNLOAD) return candidateFirmwareKey === desktopKeyForThisHost(); + // A desktop's own program installs on that desktop only; a board the web installer flashes takes its own firmware. + if (isDesktopProgram(candidateFirmwareKey)) return ownFirmwareKey === candidateFirmwareKey; if (!ownFirmwareKey) return true; - // A desktop reports "unknown": there is no ESP32 variant to name. Match it to the archive built - // for the OS the browser is running on, so a Mac is offered the .dmg and not the Windows .zip. - if (ownFirmwareKey === "unknown") return candidateFirmwareKey === desktopKeyForThisHost(); const strip = f => f.replace(/-eth.*$/, ""); return strip(ownFirmwareKey) === strip(candidateFirmwareKey); } @@ -423,7 +434,7 @@ function render(state) { // Row order: Release → Device → Firmware. Release first because it's the version the user wants to flash (the picker's primary identity); Device second so the firmware narrowing happens in front of Firmware; Firmware last so it shows the narrowed list immediately below the device that filtered it. // The device row only renders when (a) the caller didn't opt out and (b) the catalog loaded: on-device OTA passes enableDevicePicker:false, since a device flashing itself already knows what it is, and a catalog that failed to load on the web installer (rare) falls back to a two-row Release+Firmware layout with no narrowing. // In desktop mode the target is the computer viewing the page rather than a device, so there is no catalog to narrow by. The web installer draws its own Device row (a picture grid it hides in this mode), so naming the computer here would be a second one; only the on-device card, which has no device row of its own, gets the static label. - const desktopMode = state.ownFirmwareKey === "unknown"; + const desktopMode = state.ownFirmwareKey === DESKTOP_DOWNLOAD; const deviceRow = (desktopMode && !state.enableDevicePicker) ? `
Device @@ -565,7 +576,7 @@ function render(state) { // the web installer with no ownFirmwareKey accepts every candidate, which would otherwise // offer "macOS arm64" as something to flash onto an ESP32. The target picked in the Port // row decides which kind is on offer. - const wantDesktop = state.ownFirmwareKey === "unknown"; + const wantDesktop = state.ownFirmwareKey === DESKTOP_DOWNLOAD; let compatible = (r.firmwares || []) .filter(f => !!f.isDesktop === wantDesktop) // A MoonBase key is a CHIP ("esp32s3"), not a firmware variant, so the variant rule @@ -586,8 +597,7 @@ function render(state) { } if (compatible.length === 0) { // Distinguish the no-match reasons: - // - device firmware "unknown" (build didn't propagate - // MM_FIRMWARE_NAME) → a build bug; surface it for the dev. + // - download mode with no archive for this OS. // - a device whose firmware exists in ANOTHER release we can see // but not the selected one → a newer firmware variant (e.g. a // device added after the last stable). Point the user at the @@ -596,9 +606,7 @@ function render(state) { // lands in `latest` / the next tag, not the older stable. // - otherwise → genuinely no build for this device/firmware. let reason; - if (state.ownFirmwareKey === "unknown") { - // "unknown" on a DESKTOP is correct, not a build bug: there is no ESP32 variant. - // It only reaches here when the release ships no archive for this OS. + if (state.ownFirmwareKey === DESKTOP_DOWNLOAD) { reason = desktopKeyForThisHost() ? "this release has no desktop build for your OS" : "no desktop build is packaged for your OS: build from source"; @@ -831,18 +839,17 @@ export const installPicker = { * desktop build for the machine viewing the page. The port dropdown offers "This * computer" as an entry alongside the USB ports, and picking it lands here. * - * Reuses the on-device desktop path rather than adding a parallel one: a desktop - * reports its firmware key as "unknown", which isCompatible() already maps to this - * host's archive, so the labels, the Download button and the download itself all - * follow with no second implementation. The device picker is hidden while it is on, - * because no device is being flashed. + * The same download mode a desktop's Firmware card uses when its host or its user updates + * it: isCompatible() maps DESKTOP_DOWNLOAD to this host's archive, so the labels, the + * Download button and the download itself follow with no second implementation. The + * device picker is hidden while it is on, because no device is being flashed. */ setDesktopMode(on) { const state = _lastState; // Called before init() finished mounting: remember it, and init applies it once the // state exists. Without this the host's opening default is silently dropped. if (!state) { _pendingDesktopMode = on; return; } - const want = on ? "unknown" : null; + const want = on ? DESKTOP_DOWNLOAD : null; if (state.ownFirmwareKey === want) return; state.ownFirmwareKey = want; render(state); @@ -927,7 +934,7 @@ export const installPicker = { _lastState = state; // Honour a mode the host chose while the release fetch was still in flight. if (_pendingDesktopMode !== null) { - if (_pendingDesktopMode) state.ownFirmwareKey = "unknown"; + if (_pendingDesktopMode) state.ownFirmwareKey = DESKTOP_DOWNLOAD; _pendingDesktopMode = null; } if (!data) { diff --git a/test/CMakeLists.txt b/test/CMakeLists.txt index 7075bf799..cdb6e733c 100644 --- a/test/CMakeLists.txt +++ b/test/CMakeLists.txt @@ -121,6 +121,7 @@ add_executable(mm_tests unit/core/unit_platform_hdmi.cpp unit/core/unit_platform_worker.cpp unit/core/unit_platform_httpfetch.cpp + unit/core/unit_platform_update.cpp unit/core/unit_HostList.cpp unit/core/unit_Addressing.cpp unit/core/unit_HostResolver.cpp @@ -134,6 +135,7 @@ add_executable(mm_tests unit/core/unit_MoonCloudModule.cpp unit/core/unit_sha1.cpp unit/core/unit_sha256.cpp + unit/core/unit_ProgramImage.cpp unit/core/unit_MoonStatsReport.cpp unit/core/unit_SystemModule.cpp unit/core/unit_Services.cpp diff --git a/test/js/installer-desktop-download.test.mjs b/test/js/installer-desktop-download.test.mjs index 638562754..e08d28d9d 100644 --- a/test/js/installer-desktop-download.test.mjs +++ b/test/js/installer-desktop-download.test.mjs @@ -14,7 +14,7 @@ import { readFileSync } from "node:fs"; import { dirname, join } from "node:path"; import { fileURLToPath } from "node:url"; -import { parseFirmwaresFromAssets, isCompatible } from "../../src/ui/install-picker.js"; +import { parseFirmwaresFromAssets, isCompatible, DESKTOP_DOWNLOAD } from "../../src/ui/install-picker.js"; const ROOT = join(dirname(fileURLToPath(import.meta.url)), "..", ".."); const asset = (name) => ({ name, browser_download_url: `https://gh/${name}` }); @@ -122,11 +122,11 @@ test("an ESP32 device is never offered a desktop archive to flash", () => { } }); -test("a desktop is never offered an ESP32 firmware as its update", () => { - // A desktop build has no ESP32 variant to name, so it reports "unknown". Whichever OS - // the page runs on, no ESP32 image is ever a desktop upgrade. +test("a desktop is never offered an ESP32 firmware, downloading or updating itself", () => { + // Whichever OS the page runs on, no ESP32 image is a desktop's download or its update. for (const key of ["esp32", "esp32s3-n16r8", "esp32p4", "esp32-eth"]) { - assert.equal(isCompatible("unknown", key), false, `unknown must reject ${key}`); + assert.equal(isCompatible(DESKTOP_DOWNLOAD, key), false, `download mode must reject ${key}`); + assert.equal(isCompatible("desktop-linux-arm64", key), false, `a desktop must reject ${key}`); } }); @@ -135,7 +135,7 @@ test("a desktop is offered exactly one archive: the one built for its own OS", ( // of them: exactly one of the three desktop keys matches, never two, never none. That // is what stops a Mac being offered the Windows .zip. const keys = ["desktop-macos-arm64", "desktop-windows-x64", "desktop-linux-x64"]; - const matched = keys.filter(k => isCompatible("unknown", k)); + const matched = keys.filter(k => isCompatible(DESKTOP_DOWNLOAD, k)); assert.equal(matched.length, 1, `expected exactly one match, got ${matched}`); }); diff --git a/test/js/ui-update-asset.test.mjs b/test/js/ui-update-asset.test.mjs new file mode 100644 index 000000000..38947f496 --- /dev/null +++ b/test/js/ui-update-asset.test.mjs @@ -0,0 +1,52 @@ +// Which release image updates a device: firmware--v.bin for a board and a desktop alike, with the digest GitHub names; a device its +// host or its user updates gets a null address. Runs in node, on the function's own source. +// Run: `node --test test/js/ui-update-asset.test.mjs`. + +import { test } from "node:test"; +import assert from "node:assert/strict"; + +import { fnSource } from "./app-source.mjs"; +import { isCompatible, parseFirmwaresFromAssets } from "../../src/ui/install-picker.js"; + +const { updateAsset } = new Function(`${fnSource("updateAsset")}\nreturn { updateAsset };`)(); + +const release = { + assets: [ + { name: "firmware-esp32s3-n16r8-v6.2.0.bin", browser_download_url: "https://x/fw.bin", digest: "sha256:aa" }, + { name: "firmware-desktop-linux-arm64-v6.2.0.bin", browser_download_url: "https://x/pi.bin", digest: "sha256:bb" }, + ], +}; + +test("a board and a desktop each install their own image by the same name", () => { + assert.deepEqual(updateAsset(release, { firmware: "esp32s3-n16r8", update: "self" }, "6.2.0"), { url: "https://x/fw.bin", sha256: "aa" }); + assert.deepEqual(updateAsset(release, { firmware: "desktop-linux-arm64", update: "self" }, "6.2.0"), { url: "https://x/pi.bin", sha256: "bb" }); +}); + +test("a board on firmware older than the update field installs as before", () => { + assert.equal(updateAsset(release, { firmware: "esp32s3-n16r8" }, "6.2.0").url, "https://x/fw.bin"); +}); + +test("a container, a program in a read-only folder and a desktop older than its firmware name get a null address", () => { + assert.equal(updateAsset(release, { firmware: "desktop-linux-arm64", update: "host" }, "6.2.0").url, null); + assert.equal(updateAsset(release, { firmware: "desktop-linux-arm64", update: "manual" }, "6.2.0").url, null); + assert.equal(updateAsset(release, { firmware: null }, "6.2.0").url, null); +}); + +test("a desktop's own program is offered to that desktop only, a board the web installer flashes taking its own firmware", () => { + assert.equal(isCompatible("desktop-linux-arm64", "desktop-linux-arm64"), true); + assert.equal(isCompatible("desktop-linux-x64", "desktop-linux-arm64"), false); + assert.equal(isCompatible(null, "desktop-linux-arm64"), false); + assert.equal(isCompatible("esp32", "desktop-linux-arm64"), false); +}); + +test("a release lists a desktop's own program beside its download of the same name, one to install and one to save", () => { + const parsed = parseFirmwaresFromAssets([ + { name: "firmware-desktop-linux-arm64-v6.2.0.bin", browser_download_url: "https://x/pi.bin", digest: "sha256:bb" }, + { name: "moonlight_6.2.0_arm64.deb", browser_download_url: "https://x/pi.deb" }, + ], "v6.2.0"); + const program = parsed.find(f => f.firmware === "desktop-linux-arm64" && !f.isDesktop); + const download = parsed.find(f => f.firmware === "desktop-linux-arm64" && f.isDesktop); + assert.equal(program?.binaryUrl, "https://x/pi.bin"); + assert.equal(program?.sha256, "bb"); // the digest the Firmware card posts with the address, as the map does + assert.equal(download?.binaryUrl, "https://x/pi.deb"); +}); diff --git a/test/python/test_package_deb.py b/test/python/test_package_deb.py new file mode 100644 index 000000000..ff4b62de9 --- /dev/null +++ b/test/python/test_package_deb.py @@ -0,0 +1,63 @@ +# /// script +# dependencies = ["pytest"] +# /// +"""The .deb's layout: the program as a seed under /usr/lib/moonlight, the service and its rollback, the maintainer scripts that install them, and a menu entry opening the page. +Read from the staged tree at the moment dpkg-deb would pack it, so it runs on any machine.""" + +import sys +from pathlib import Path + +sys.path.insert(0, str(Path(__file__).resolve().parents[2] / "moondeck" / "ci")) +import package_desktop as pd # noqa: E402 + + +def staged(tmp_path, monkeypatch): + """The files package_deb stages, relative to the package root, with their modes.""" + binary = tmp_path / "MoonLight" + binary.write_text("program") + monkeypatch.setattr(pd, "DIST_DIR", tmp_path / "dist") + monkeypatch.setattr(pd.shutil, "which", lambda name: "/usr/bin/dpkg-deb") + seen = {} + + def pack(cmd, *args, **kwargs): + stage = Path(cmd[-2]) + seen.update({p.relative_to(stage).as_posix(): p.stat().st_mode & 0o777 for p in stage.rglob("*") if p.is_file()}) + seen["_desktop"] = (stage / "usr/share/applications/moonlight.desktop").read_text() + + monkeypatch.setattr(pd, "run", pack) + pd.package_deb(binary, "6.2.0-dev.3") + return seen + + +def test_the_program_is_a_seed_the_service_copies_into_its_own_folder(tmp_path, monkeypatch): + files = staged(tmp_path, monkeypatch) + assert "usr/lib/moonlight/MoonLight" in files + assert "usr/bin/MoonLight" not in files + + +def test_the_package_carries_the_service_its_rollback_and_the_scripts_that_install_them(tmp_path, monkeypatch): + files = staged(tmp_path, monkeypatch) + for unit in ("usr/lib/systemd/system/moonlight.service", "usr/lib/systemd/system/moonlight-rollback.service"): + assert unit in files + for script in ("DEBIAN/postinst", "DEBIAN/prerm", "DEBIAN/postrm", "usr/lib/moonlight/rollback"): + assert files[script] == 0o755, script + + +def test_the_menu_entry_opens_the_running_service_page(tmp_path, monkeypatch): + desktop = staged(tmp_path, monkeypatch)["_desktop"] + assert "Exec=xdg-open http://localhost:8080/" in desktop + assert "Terminal=false" in desktop + + +def test_the_service_runs_the_copy_its_user_owns_and_rolls_back_through_systemd(): + unit = (pd.LINUX_DIR / "moonlight.service").read_text() + assert "ExecStart=/var/lib/moonlight/bin/MoonLight" in unit + assert "User=moonlight" in unit + assert "Restart=always" in unit + assert "OnFailure=moonlight-rollback.service" in unit + for confinement in ("NoNewPrivileges=yes", "ProtectSystem=strict", "ReadWritePaths=/var/lib/moonlight"): + assert confinement in unit, confinement + # One capability, the raw frames a ColorLight card takes, granted and bounded alike. + assert "AmbientCapabilities=CAP_NET_RAW\n" in unit and "CapabilityBoundingSet=CAP_NET_RAW\n" in unit + postinst = (pd.LINUX_DIR / "postinst").read_text() + assert "/usr/lib/moonlight/MoonLight" in postinst and "/var/lib/moonlight/bin" in postinst diff --git a/test/python/test_release_stage.py b/test/python/test_release_stage.py index c04a10b77..9275f83f3 100644 --- a/test/python/test_release_stage.py +++ b/test/python/test_release_stage.py @@ -27,6 +27,10 @@ def test_a_push_to_main_releases_as_latest(): assert rs.resolve_tag("", "refs/heads/main", "main") == "latest" +def test_a_pull_request_builds_as_the_next_latest_never_as_a_stable_version(): + assert rs.resolve_tag("", "refs/pull/250/merge", "250/merge") == "latest" + + def test_a_version_tag_push_releases_under_its_own_name(): assert rs.resolve_tag("", "refs/tags/v7.1.0-rc1", "v7.1.0-rc1") == "v7.1.0-rc1" diff --git a/test/unit/core/unit_FirmwareUpdateModule.cpp b/test/unit/core/unit_FirmwareUpdateModule.cpp index 5bff24dd4..8812e79c5 100644 --- a/test/unit/core/unit_FirmwareUpdateModule.cpp +++ b/test/unit/core/unit_FirmwareUpdateModule.cpp @@ -70,11 +70,13 @@ TEST_CASE("FirmwareUpdateModule OTA status routes through the status slot") { } namespace { -const char* readOnly(mm::FirmwareUpdateModule& fw, const char* name) { +/// A control's storage by its name, so a test reads and sets it whatever controls come before it. +void* controlPtr(mm::FirmwareUpdateModule& fw, const char* name) { for (uint8_t i = 0; i < fw.controls().count(); i++) - if (std::strcmp(fw.controls()[i].name, name) == 0) return static_cast(fw.controls()[i].ptr); + if (std::strcmp(fw.controls()[i].name, name) == 0) return fw.controls()[i].ptr; return nullptr; } +const char* readOnly(mm::FirmwareUpdateModule& fw, const char* name) { return static_cast(controlPtr(fw, name)); } } // namespace // MoonBase's `+` splits into its version and build rows; a mismatch with the app warns between installs and is published for the UI's signals. @@ -90,7 +92,8 @@ TEST_CASE("a MoonBase that does not match the app says so, and only then") { CHECK(std::strcmp(readOnly(fw, "appVersion"), mm::kVersion) == 0); REQUIRE(readOnly(fw, "moonbaseMismatch") != nullptr); CHECK(std::strcmp(readOnly(fw, "moonbaseMismatch"), "0.0.1") == 0); - *static_cast(fw.controls()[2].ptr) = 1; // the MoonBase tab: appVersion, moonbaseMismatch, image + REQUIRE(controlPtr(fw, "image") != nullptr); + *static_cast(controlPtr(fw, "image")) = 1; // the MoonBase tab fw.rebuildControls(); CHECK(std::strcmp(readOnly(fw, "version"), "0.0.1") == 0); CHECK(std::strcmp(readOnly(fw, "build"), "abc12345+ · Oct 4 2026 10:31:00") == 0); diff --git a/test/unit/core/unit_HttpServerModule_apply.cpp b/test/unit/core/unit_HttpServerModule_apply.cpp index 9936aca4f..9ae70346f 100644 --- a/test/unit/core/unit_HttpServerModule_apply.cpp +++ b/test/unit/core/unit_HttpServerModule_apply.cpp @@ -798,7 +798,7 @@ TEST_CASE("the frame walker hands a Ping and a Close to the control callback, in CHECK(seen[2] == "8:\x03\xe8"); } -TEST_CASE("/api/system carries the summary a peer's map shows: identity, board, signal, light outputs and effects") { +TEST_CASE("/api/system carries the summary a peer's map shows: how it updates, identity, board, signal, light outputs and effects") { PatchTree t; struct Lights final : mm::test::FakeLightOutput {} lights; lights.outputs_ = "{\"driver\":\"LedDriver\",\"lights\":300}"; @@ -810,6 +810,8 @@ TEST_CASE("/api/system carries the summary a peer's map shows: identity, board, t.http.writeDeviceSummaryForTest(sink); } const std::string doc = std::string("{") + buf + "\"end\":0}"; + // The mode itself is pinned in unit_platform_update.cpp; here only that the summary names one. + CHECK(doc.find("\"update\":\"") != std::string::npos); CHECK(doc.find("\"mac\":\"") != std::string::npos); CHECK(doc.find("\"board\":") != std::string::npos); CHECK(doc.find("\"chip\":") != std::string::npos); diff --git a/test/unit/core/unit_HttpServerModule_routes.cpp b/test/unit/core/unit_HttpServerModule_routes.cpp index 9dd1b677c..373ea4335 100644 --- a/test/unit/core/unit_HttpServerModule_routes.cpp +++ b/test/unit/core/unit_HttpServerModule_routes.cpp @@ -139,6 +139,22 @@ TEST_CASE("a firmware upload without a Content-Length is refused with 411, as ev } // A name past what a module holds is refused on every route that names one, rather than cut short into a name that could match another module. +TEST_CASE("an image URL longer than the MoonBase handoff takes is refused with 400 rather than fetched cut short") { + mm::HttpServerModule http; + http.port = kTestPort; + http.setup(); + const std::string longUrl = "https://example.com/" + std::string(300, 'a') + ".bin"; + mm::platform::TcpConnection client; + REQUIRE(connectAndAccept(http, client, kTestPort)); + const std::string req = withBody("POST /api/firmware/url", "{\"url\":\"" + longUrl + "\"}"); + REQUIRE(client.write(reinterpret_cast(req.data()), req.size())); + const std::string reply = readUntilClosed(http, client); // the whole answer: the reason follows the status line + client.close(); + CHECK(reply.rfind("HTTP/1.1 400", 0) == 0); + CHECK(reply.find("url too long") != std::string::npos); + http.release(); +} + TEST_CASE("a module route whose name does not fit is refused with 400 on move, replace, delete and read alike") { RackServer server; mm::HttpServerModule& http = server.http; diff --git a/test/unit/core/unit_ProgramImage.cpp b/test/unit/core/unit_ProgramImage.cpp new file mode 100644 index 000000000..2ab7988c0 --- /dev/null +++ b/test/unit/core/unit_ProgramImage.cpp @@ -0,0 +1,70 @@ +/// @module core + +/// Pins what a desktop program's first bytes say about it, the check a desktop runs before a new version replaces it, as a board checks its image's chip. +/// +/// An ELF, Mach-O or PE header for the right system and processor passes, another one is refused with its reason, and a web page or an archive is no program. + +#include "doctest.h" +#include "core/util/ProgramImage.h" + +#include +#include +#include + +using namespace mm; + +namespace { + +void putLe16(std::vector& b, size_t at, uint16_t v) { b[at] = static_cast(v); b[at + 1] = static_cast(v >> 8); } +void putLe32(std::vector& b, size_t at, uint32_t v) { for (int i = 0; i < 4; i++) b[at + i] = static_cast(v >> (8 * i)); } + +std::vector elf(uint16_t machine) { + std::vector b(64, 0); + b[0] = 0x7F; b[1] = 'E'; b[2] = 'L'; b[3] = 'F'; b[4] = 2; + putLe16(b, 18, machine); + return b; +} + +std::vector machO(uint32_t cpu) { + std::vector b(32, 0); + putLe32(b, 0, 0xFEEDFACFu); + putLe32(b, 4, cpu); + return b; +} + +std::vector pe(uint16_t machine) { + std::vector b(0x100, 0); + b[0] = 'M'; b[1] = 'Z'; + putLe32(b, 0x3C, 0x80); + putLe32(b, 0x80, 0x00004550u); + putLe16(b, 0x84, machine); + return b; +} + +} // namespace + +TEST_CASE("a program for the right system and processor passes, in each format") { + CHECK(programRejection(elf(62), ProgramSystem::Linux, ProgramCpu::X64) == nullptr); + CHECK(programRejection(elf(183), ProgramSystem::Linux, ProgramCpu::Arm64) == nullptr); + CHECK(programRejection(machO(0x0100000Cu), ProgramSystem::MacOS, ProgramCpu::Arm64) == nullptr); + CHECK(programRejection(machO(0x01000007u), ProgramSystem::MacOS, ProgramCpu::X64) == nullptr); + CHECK(programRejection(pe(0x8664), ProgramSystem::Windows, ProgramCpu::X64) == nullptr); + CHECK(programRejection(pe(0xAA64), ProgramSystem::Windows, ProgramCpu::Arm64) == nullptr); +} + +TEST_CASE("a program for another system or another processor is refused with that reason") { + CHECK(std::string(programRejection(pe(0x8664), ProgramSystem::Linux, ProgramCpu::X64)) == "a program for another system"); + CHECK(std::string(programRejection(elf(62), ProgramSystem::MacOS, ProgramCpu::Arm64)) == "a program for another system"); + CHECK(std::string(programRejection(elf(62), ProgramSystem::Linux, ProgramCpu::Arm64)) == "a program for another processor"); + CHECK(std::string(programRejection(machO(0x01000007u), ProgramSystem::MacOS, ProgramCpu::Arm64)) == "a program for another processor"); +} + +TEST_CASE("a web page, an archive or a truncated header is no program") { + const std::string page = "404"; + CHECK(std::string(programRejection(std::vector(page.begin(), page.end()), ProgramSystem::Linux, ProgramCpu::X64)) == "not a program"); + CHECK(std::string(programRejection(std::vector{0x1F, 0x8B, 0x08, 0x00}, ProgramSystem::MacOS, ProgramCpu::Arm64)) == "not a program"); // gzip + std::vector shortPe = pe(0x8664); + shortPe.resize(0x82); // the PE signature's offset points past the end + CHECK(std::string(programRejection(shortPe, ProgramSystem::Windows, ProgramCpu::X64)) == "not a program"); + CHECK(std::string(programRejection({}, ProgramSystem::Windows, ProgramCpu::X64)) == "not a program"); +} diff --git a/test/unit/core/unit_platform_update.cpp b/test/unit/core/unit_platform_update.cpp new file mode 100644 index 000000000..0fe0098d4 --- /dev/null +++ b/test/unit/core/unit_platform_update.cpp @@ -0,0 +1,164 @@ +/// @module platform + +/// Pins a desktop installing a new version of itself as a board does, each test over a scratch copy standing in for the running program. +/// +/// An image written through `otaBegin`, `otaWrite` and `otaEnd` replaces a program of this system and processor, the old one kept as `.prev`, the new one `.pending`. +/// +/// An image that is no program, misses its digest or is abandoned leaves the program as it was; a program in a folder it cannot write is updated by hand. + +#include "doctest.h" +#include "platform/platform.h" +#include "host_update.h" +#include "core/util/sha256.h" + +#include +#include +#include +#include + +#ifndef _WIN32 +#include // geteuid +#endif + +using namespace mm; +namespace fs = std::filesystem; + +namespace { + +// FILE rather than a stream: a stream's object takes more stack than the frame limit allows. +std::vector readBytes(const fs::path& path) { + std::vector bytes; + std::FILE* file = std::fopen(path.string().c_str(), "rb"); + if (!file) return bytes; + uint8_t chunk[256]; + for (size_t n; (n = std::fread(chunk, 1, sizeof(chunk), file)) > 0;) bytes.insert(bytes.end(), chunk, chunk + n); + std::fclose(file); + return bytes; +} + +void writeBytes(const fs::path& path, const std::vector& bytes) { + std::FILE* file = std::fopen(path.string().c_str(), "wb"); + REQUIRE(file != nullptr); + std::fwrite(bytes.data(), 1, bytes.size(), file); + std::fclose(file); +} + +void writeText(const fs::path& path, const std::string& text) { writeBytes(path, std::vector(text.begin(), text.end())); } + +std::string readText(const fs::path& path) { + const std::vector bytes = readBytes(path); + return {bytes.begin(), bytes.end()}; +} + +/// The bytes' SHA-256 in capitals: GitHub writes lowercase, and either case is the same digest. +std::string upperDigestOf(const std::vector& bytes) { + char hex[kSha256DigestSize * 2 + 1]; + sha256Hex(bytes.data(), bytes.size(), hex, kSha256DigestSize); + std::string upper(hex); + for (char& c : upper) c = static_cast(c >= 'a' && c <= 'f' ? c - 'a' + 'A' : c); + return upper; +} + +/// A folder holding a stand-in program, "old program", which every install in it replaces; the real program path comes back after the test. +struct Scratch { + fs::path dir; + fs::path program; + std::vector realProgram; ///< this test binary's bytes: a program for this system and processor, the image a release would carry + explicit Scratch(const char* name) { + realProgram = readBytes(platform::programPath()); + dir = fs::temp_directory_path() / (std::string("mm_update_") + name); + fs::remove_all(dir); + fs::create_directories(dir); + program = dir / "MoonLight"; + writeText(program, "old program"); + platform::setTestProgramPath(program); + } + ~Scratch() { + platform::setTestProgramPath({}); + std::error_code ec; + fs::permissions(dir, fs::perms::owner_all, ec); + fs::remove_all(dir, ec); + } + Scratch(const Scratch&) = delete; + Scratch& operator=(const Scratch&) = delete; + /// Write `image` through the image writer in chunks, as the upload route does, and commit it or not. + bool install(const std::vector& image, bool commit = true) const { + char status[64] = {}; + uint32_t written = 0; + platform::OtaWrite w; + if (!platform::otaBegin(w, platform::OtaTarget::App, image.size(), status, sizeof(status), &written)) return false; + for (size_t at = 0; at < image.size(); at += 4096) { + const size_t n = image.size() - at < 4096 ? image.size() - at : 4096; + if (!platform::otaWrite(w, image.data() + at, n)) return false; + } + CHECK(written == image.size()); + return platform::otaEnd(w, commit); + } +}; + +} // namespace + +TEST_CASE("a native program in a folder it can write updates itself, one in a folder it cannot write is updated by hand") { + Scratch s("mode"); + CHECK(platform::updateMode() == platform::UpdateMode::Self); +#ifndef _WIN32 + // Root may write any folder whatever its mode, so this half holds for an ordinary user only. + if (::geteuid() != 0) { + fs::permissions(s.dir, fs::perms::owner_read | fs::perms::owner_exec); + CHECK(platform::updateMode() == platform::UpdateMode::Manual); + } +#endif + CHECK(std::string(platform::updateModeName(platform::UpdateMode::Self)) == "self"); + CHECK(std::string(platform::updateModeName(platform::UpdateMode::Host)) == "host"); + CHECK(std::string(platform::updateModeName(platform::UpdateMode::Manual)) == "manual"); +} + +TEST_CASE("an image written through the image writer replaces the program and keeps the old one as .prev, pending until it has run") { + Scratch s("install"); + REQUIRE(s.install(s.realProgram)); + CHECK(readBytes(s.program) == s.realProgram); + CHECK(readText(fs::path(s.program.string() + ".prev")) == "old program"); + CHECK(fs::exists(s.program.string() + ".pending")); + CHECK_FALSE(fs::exists(s.dir / "MoonLight.next")); +} + +TEST_CASE("an image that is no program for this system leaves the program as it was") { + Scratch s("refused"); + const std::string page = "404 Not Found"; + CHECK_FALSE(s.install(std::vector(page.begin(), page.end()))); + CHECK(readText(s.program) == "old program"); + CHECK_FALSE(fs::exists(s.program.string() + ".prev")); + CHECK_FALSE(fs::exists(s.dir / "MoonLight.next")); +} + +TEST_CASE("an abandoned image leaves the program as it was") { + Scratch s("abandoned"); + CHECK_FALSE(s.install(s.realProgram, false)); + CHECK(readText(s.program) == "old program"); + CHECK_FALSE(fs::exists(s.dir / "MoonLight.next")); +} + +TEST_CASE("an image missing the release's digest leaves the program as it was, and one matching it in either case is installed") { + Scratch s("digest"); + const fs::path next = s.dir / "MoonLight.next"; + writeBytes(next, s.realProgram); + char status[64] = {}; + const std::string wrong(kSha256DigestSize * 2, 'a'); + CHECK_FALSE(platform::putProgramInPlace(next, s.program, wrong.c_str(), status, sizeof(status))); + CHECK(std::string(status).find("digest") != std::string::npos); + CHECK(readText(s.program) == "old program"); + + writeBytes(next, s.realProgram); + CHECK(platform::putProgramInPlace(next, s.program, upperDigestOf(s.realProgram).c_str(), status, sizeof(status))); + CHECK(readBytes(s.program) == s.realProgram); +} + +TEST_CASE("a desktop has one program: MoonBase and the radio chip are refused") { + Scratch s("target"); + char status[64] = {}; + uint32_t written = 0; + platform::OtaWrite w; + CHECK_FALSE(platform::otaBegin(w, platform::OtaTarget::MoonBase, 0, status, sizeof(status), &written)); + CHECK_FALSE(platform::otaBegin(w, platform::OtaTarget::Coprocessor, 0, status, sizeof(status), &written)); + CHECK(std::string(status).rfind("error:", 0) == 0); +} From cc73970685d161428776386dd1357202c0b51b61 Mon Sep 17 00:00:00 2001 From: ewowi Date: Fri, 13 Nov 2026 11:04:46 +0100 Subject: [PATCH 2/5] Close the gaps CodeRabbit found in the desktop update A Linux desktop now restarts on its new version rather than exiting, a library or an object file is refused as an update, and an install from a release carrying both the package and the program installs the program. The install overlay closes as soon as a restarted desktop answers. KPI: 256lights | Desktop:2544KB | tick:8/14/12/3/12/13/306/20/10/10us(FPS:125000/71428/83333/333333/83333/76923/3267/50000/100000/100000) | ESP32:2316KB | src:393(99502) | test:292(70145) | functions-over:333 Commit: src +67/-39 lines | test +48/-17 lines | duplicated lines 3186 -> 3186 (+0) **Core** - ProgramImage: a header is read whole and must be a program the loader starts: ELF ET_EXEC or a PIE, Mach-O MH_EXECUTE, PE an executable image that is no DLL. - Desktop update: the program's path is read once at start, since Linux names a replaced program "(deleted)"; the .pending marker is written before the swap and removed on a failed one; a failed close reports its error; the update stays "downloading" while the fallback address is still to try. - The encoder's pipes close on exec, so a restart in place ends the old encoder; a failed WinHTTP read fails the transfer. - CMake names the processor from CMAKE_OSX_ARCHITECTURES when it names one. **UI** - pickedEntry: the install takes the picked firmware of the kind on offer, as the dropdown lists it. watchInstall: back to rest after work is success for every install, so a desktop restarting between polls closes the overlay. **Scripts/MoonDeck** - postinst restarts the service on configure only. The Dockerfile finds the program wherever the release's package keeps it, so every tag builds. **Tests** - unit_ProgramImage: a fixed-address program passes, a library or an object file is refused, a cut ELF header is no program. The JS picker test installs the program with both kinds in one release. The self-update tests skip in a container; the test helper writes through a stream. - Desktop 3269/3269, JS 397, Python packaging 23. - On this Mac, tested by the product owner: an upload installs and restarts in place (same PID) and .pending clears after a minute; a board image is refused as not a program; a URL with a wrong digest is refused, with the right one it installs and restarts in place. **Docs/CI** - 02-install: the package service's settings path; the desktop update step says a program updates itself. **Reviews** - CodeRabbit, the Linux restart exiting on a "(deleted)" path: done, the path is cached at start. - CodeRabbit, an ELF object accepted: done, every format checks its executable type. - CodeRabbit, a second update admitted between the two download attempts: done. - CodeRabbit, the .pending marker unchecked: done, written first. - CodeRabbit, a failed close left "flashing": done. - CodeRabbit, encoder pipes surviving execv: done, close-on-exec. - CodeRabbit, a broken WinHTTP read counted as complete: done. - CodeRabbit, install picking the .deb over the program: done, pinned in a test that fails against the old lookup. - CodeRabbit, the overlay missing a fast restart: done. - CodeRabbit, the Apple target architecture: done. - CodeRabbit, the Dockerfile on an older release: done, the program is found rather than named; a release-first sequence would leave every older tag unbuildable. - CodeRabbit, postinst acting on abort: done. - CodeRabbit, the tests failing in a container: done, skipped there. - CodeRabbit, the settings path and the update step in 02-install: done. - CodeQL, a test file created with mode 0666: done. - GCC compile-proof build: SKIP, the Linux encoder change and the Windows read loop compile in the PR's CI. Co-Authored-By: Claude Opus 5.5 --- CMakeLists.txt | 8 +++- Dockerfile | 8 +++- docs/reference/metrics/repo-health.json | 28 ++++++------- docs/reference/metrics/repo-health.md | 42 +++++++++---------- moondeck/ci/linux/postinst | 12 +++--- moonwalk/02-install.yaml | 8 ++-- src/core/util/ProgramImage.h | 29 +++++++------ src/platform/desktop/platform_desktop.cpp | 15 +++++-- .../desktop/platform_desktop_update.cpp | 36 +++++++++++----- src/ui/app.js | 14 ++----- src/ui/install-picker.js | 12 +++++- test/js/installer-desktop-download.test.mjs | 13 +++++- .../core/unit_HttpServerModule_routes.cpp | 6 ++- test/unit/core/unit_ProgramImage.cpp | 22 ++++++++-- test/unit/core/unit_platform_update.cpp | 24 ++++++----- 15 files changed, 173 insertions(+), 104 deletions(-) diff --git a/CMakeLists.txt b/CMakeLists.txt index ada2ce00e..ac5372b85 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -181,7 +181,13 @@ elseif(WIN32) else() set(MM_DESKTOP_OS linux) endif() -string(TOLOWER "${CMAKE_SYSTEM_PROCESSOR}" MM_DESKTOP_CPU) +# The processor the program is built for: on Apple, CMAKE_OSX_ARCHITECTURES when it names one, since CMAKE_SYSTEM_PROCESSOR may name the host's. +set(MM_DESKTOP_CPU "${CMAKE_SYSTEM_PROCESSOR}") +list(LENGTH CMAKE_OSX_ARCHITECTURES MM_OSX_ARCH_COUNT) +if(APPLE AND MM_OSX_ARCH_COUNT EQUAL 1) + set(MM_DESKTOP_CPU "${CMAKE_OSX_ARCHITECTURES}") +endif() +string(TOLOWER "${MM_DESKTOP_CPU}" MM_DESKTOP_CPU) if(MM_DESKTOP_CPU MATCHES "^(arm64|aarch64)$") set(MM_DESKTOP_CPU arm64) elseif(MM_DESKTOP_CPU MATCHES "^(x86_64|amd64)$") diff --git a/Dockerfile b/Dockerfile index d1e2b6527..dee7dbe66 100644 --- a/Dockerfile +++ b/Dockerfile @@ -75,6 +75,10 @@ RUN apt-get update \ && test -n "$url" || { echo "no ${TARGETARCH} .deb in release ${RELEASE}" >&2; exit 1; } \ && curl -fsSL -o /tmp/moonlight.deb "$url" \ && dpkg-deb -x /tmp/moonlight.deb /rootfs \ + # The program wherever the release's package keeps it, found rather than named, so every tag builds whatever its layout. + && program=$(find /rootfs/usr -type f -name MoonLight | head -1) \ + && test -n "$program" \ + && mv "$program" /MoonLight \ # Every shared object the binary resolves to, gathered by asking the loader rather than by # listing names: ldd walks the whole transitive chain, so this stays correct as that chain moves. # The four the distroless base already carries (libc, libstdc++, libm, libgcc_s) are EXCLUDED @@ -84,7 +88,7 @@ RUN apt-get update \ && pulse=$(find /usr/lib -name libpulse.so.0 | head -1) \ && test -n "$pulse" \ # Captured first, so a failing ldd, or one naming a library it cannot find, stops the build here. - && ldd /rootfs/usr/lib/moonlight/MoonLight > /tmp/deps \ + && ldd /MoonLight > /tmp/deps \ && ldd "$pulse" >> /tmp/deps \ && ! grep -q "not found" /tmp/deps \ && echo "libpulse.so.0 => $pulse (0)" >> /tmp/deps \ @@ -108,7 +112,7 @@ RUN apt-get update \ # up base updates (docker buildx imagetools inspect gcr.io/distroless/cc-debian13:latest). FROM gcr.io/distroless/cc-debian13@sha256:9b615fff20e1a4fad29c2b30562580b212c7dd5e2225236735cca0070ed11c78 -COPY --from=fetch /rootfs/usr/lib/moonlight/MoonLight /usr/bin/MoonLight +COPY --from=fetch /MoonLight /usr/bin/MoonLight # The libraries the loader resolved in the fetch stage, collected there rather than named here. # Without this the container starts and dies immediately on "libcurl.so.4: cannot open shared # object file"; the release runs each image before it publishes one. diff --git a/docs/reference/metrics/repo-health.json b/docs/reference/metrics/repo-health.json index ceb27475a..7e37de804 100644 --- a/docs/reference/metrics/repo-health.json +++ b/docs/reference/metrics/repo-health.json @@ -1,7 +1,7 @@ { - "commit": "e6b2e9945", + "commit": "1f4285222", "flash": { - "esp32s3-n16r8": 2493520, + "esp32s3-n16r8": 2493552, "esp32": 2283888, "esp32p4rev1-eth": 2617552, "esp32p4rev1-eth-wifi": 2876672, @@ -14,7 +14,7 @@ "esp32p4rev3-eth": 2615456, "esp32s3-zero": 2125456, "esp32-pico": 2432432, - "desktop-macos": 2605824, + "desktop-macos": 2605920, "desktop-windows": 1925120, "esp32c3": 2405632, "desktop-linux": 2548000, @@ -841,16 +841,16 @@ } }, "loc": { - "core": 37899, + "core": 37904, "light": 35148, - "platform": 25805, - "ui": 14557, - "test": 86798, + "platform": 25826, + "ui": 14559, + "test": 86818, "moondeck": 35531 }, "comments": { "core": { - "lines": 7445, + "lines": 7447, "ratio": 0.218 }, "light": { @@ -858,15 +858,15 @@ "ratio": 0.233 }, "platform": { - "lines": 4518, + "lines": 4522, "ratio": 0.196 }, "ui": { - "lines": 3623, + "lines": 3626, "ratio": 0.264 }, "test": { - "lines": 7971, + "lines": 7974, "ratio": 0.106 }, "moondeck": { @@ -875,19 +875,19 @@ } }, "tests": { - "cases": 3260, + "cases": 3261, "scenarios": 37 }, "docs": { "md_files": 122, - "md_lines": 19773, + "md_lines": 19767, "plans_files": 14, "backlog_lines": 2835, "lessons_lines": 455, "claude_md_lines": 139 }, "complexity": { - "functions": 6309, + "functions": 6310, "over_threshold": 333, "worst_ccn": 128 } diff --git a/docs/reference/metrics/repo-health.md b/docs/reference/metrics/repo-health.md index 571937dd4..417feb075 100644 --- a/docs/reference/metrics/repo-health.md +++ b/docs/reference/metrics/repo-health.md @@ -1,6 +1,6 @@ # Repo health -Measured at `e6b2e9945`. Generated by [`moondeck/check/repo_health.py`](../../../moondeck/check/repo_health.py) on every KPI-gate run. **Do not edit by hand.** +Measured at `1f4285222`. Generated by [`moondeck/check/repo_health.py`](../../../moondeck/check/repo_health.py) on every KPI-gate run. **Do not edit by hand.** Current state only; the trend is this file's git history (`git log -p docs/reference/metrics/repo-health.md`). Nothing here fails a build: the numbers make growth visible, the judgment stays human. @@ -8,7 +8,7 @@ Current state only; the trend is this file's git history (`git log -p docs/refer | Metric | v7.0.0 | Now | Change | |---|---:|---:|---:| -| flash desktop-macos | 2,221 KB | 2,545 KB | +323.5 KB (+14.6%) | +| flash desktop-macos | 2,221 KB | 2,545 KB | +323.6 KB (+14.6%) | | flash esp32 | 2,095 KB | 2,230 KB | +135.5 KB (+6.5%) | | flash esp32-16mb | 2,343 KB | 2,210 KB | -133.1 KB (-5.7%) | | flash esp32-eth | 1,642 KB | 1,827 KB | +185.6 KB (+11.3%) | @@ -29,14 +29,14 @@ Current state only; the trend is this file's git history (`git log -p docs/refer | free heap esp32 | 33,034 KB | 33,034 KB | +0.0 KB (+0.0%) | | tick Effects_pipeline_builds_and_renders (desktop-macos) | 7 µs | 7 µs | +0 (+0.0%) | | tick Layouts_resize_reallocates_live (desktop-macos) | 10 µs | 10 µs | +0 (+0.0%) | -| lines core | 33,397 | 37,899 | +4,502 (+13.5%) | +| lines core | 33,397 | 37,904 | +4,507 (+13.5%) | | lines light | 31,681 | 35,148 | +3,467 (+10.9%) | -| lines platform | 22,993 | 25,805 | +2,812 (+12.2%) | -| lines ui | 13,043 | 14,557 | +1,514 (+11.6%) | -| lines test | 76,140 | 86,798 | +10,658 (+14.0%) | +| lines platform | 22,993 | 25,826 | +2,833 (+12.3%) | +| lines ui | 13,043 | 14,559 | +1,516 (+11.6%) | +| lines test | 76,140 | 86,818 | +10,678 (+14.0%) | | lines moondeck | 32,680 | 35,531 | +2,851 (+8.7%) | | code findings | 1,642 | 1,607 | -35 (-2.1%) | -| unit tests | 2,836 | 3,260 | +424 (+15.0%) | +| unit tests | 2,836 | 3,261 | +425 (+15.0%) | | scenarios | 36 | 37 | +1 (+2.8%) | Against the release `v7.0.0`, the fixed point a user compares with; the tables below compare against the last commit. A firmware not rebuilt since `v7.0.0` is left out, and a measure the release did not record yet, such as the code check, joins from the next one. @@ -46,7 +46,7 @@ Against the release `v7.0.0`, the fixed point a user compares with; the tables b | Target | Flash | Capacity | Used | Built | |---|---:|---:|---:|:--:| | desktop-linux | 2,488 KB | - | - | **STALE 32d** | -| desktop-macos | 2,545 KB (+22 KB) ⚠ | - | - | yes | +| desktop-macos | 2,545 KB (+96 B) ⚠ | - | - | yes | | desktop-windows | 1,880 KB | - | - | **STALE 44d** | | esp32 | 2,230 KB | 2,496 KB | 89% | carried 1d | | esp32-16mb | 2,210 KB | 4,096 KB | 54% | **STALE 14d** | @@ -58,7 +58,7 @@ Against the release `v7.0.0`, the fixed point a user compares with; the tables b | esp32p4rev1-eth-wifi | 2,809 KB | 4,096 KB | 69% | yes | | esp32p4rev3-eth | 2,554 KB | 4,096 KB | 62% | carried 2d | | esp32p4rev3-eth-wifi | 2,612 KB | 4,096 KB | 64% | **STALE 14d** | -| esp32s3-n16r8 | 2,435 KB (+496 B) ⚠ | 4,096 KB | 59% | yes | +| esp32s3-n16r8 | 2,435 KB (+32 B) ⚠ | 4,096 KB | 59% | yes | | esp32s3-n32r16 | 2,431 KB | 4,096 KB | 59% | carried 3d | | esp32s3-n8r2 | 2,194 KB | 3,072 KB | 71% | **STALE 14d** | | esp32s3-n8r8 | 2,301 KB | 3,072 KB | 75% | **STALE 14d** | @@ -73,13 +73,13 @@ Against the release `v7.0.0`, the fixed point a user compares with; the tables b | Target | Net | New | Removed | Grown | Shrunk | Other | Since | |---|---:|---:|---:|---:|---:|---:|---| -| esp32p4rev1-eth-wifi | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | `e6b2e9945` | -| esp32s3-n16r8 | +0.5 KB | +0.6 KB | -0.9 KB | +0.9 KB | -0.2 KB | -0.0 KB | `e6b2e9945` | +| esp32p4rev1-eth-wifi | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | `1f4285222` | +| esp32s3-n16r8 | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | `1f4285222` | Largest moves: - **esp32p4rev1-eth-wifi**: none -- **esp32s3-n16r8**: `mm::NetworkModule::tickIdle(unsigned long)` -494 B, `mm::platform::http_fetch_to_ota(char const*, char*, unsigned` -285 B, `mm::platform::http_fetch_to_ota(char const*, char*, unsigned` +285 B, `_ZZN2mm8moonlive13lightBuiltinsEvENKUlvE_clEv$isra$0` +217 B, `mm::HttpServerModule::readImageUrl(mm::platform::TcpConnecti` +166 B +- **esp32s3-n16r8**: `mm::ui::file1` +30 B What this commit's flash went to, symbol by symbol against the measurement at `Since`. New is new functionality, removed and shrunk are what a cleanup saved, and other is what no symbol owns, such as merged strings and alignment. A rename reads as removed plus new, which the largest moves expose. @@ -145,26 +145,26 @@ These build a bare pipeline with no optional modules, so a change here is a chan | Area | Lines | Comments | Comment share | |---|---:|---:|---:| -| core | 37,899 (+71) ⚠ | 7,445 | 21.8 % | +| core | 37,904 (+5) ⚠ | 7,447 | 21.8 % | | light | 35,148 | 7,277 | 23.3 % | -| platform | 25,805 (+468) ⚠ | 4,518 | 19.6 % (−0.2 %) ✓ | -| ui | 14,557 (−21) ✓ | 3,623 | 26.4 % | -| test | 86,798 (+322) ⚠ | 7,971 | 10.6 % | -| moondeck | 35,531 (+6) ⚠ | 4,509 | 14.6 % | +| platform | 25,826 (+21) ⚠ | 4,522 | 19.6 % | +| ui | 14,559 (+2) ⚠ | 3,626 | 26.4 % | +| test | 86,818 (+20) ⚠ | 7,974 | 10.6 % | +| moondeck | 35,531 | 4,509 | 14.6 % | ## Tests | Kind | Count | |---|---:| -| unit cases | 3,260 (+10) ✓ | +| unit cases | 3,261 (+1) ✓ | | scenarios | 37 | ## Complexity | Metric | Value | |---|---:| -| functions | 6,309 (+33) ✓ | -| over threshold | 333 (−2) ✓ | +| functions | 6,310 (+1) ✓ | +| over threshold | 333 | | worst CCN | 128 | ## Documentation @@ -172,7 +172,7 @@ These build a bare pipeline with no optional modules, so a change here is a chan | Metric | Value | |---|---:| | markdown files | 122 | -| markdown lines | 19,773 (+82) ⚠ | +| markdown lines | 19,767 (−6) ✓ | | plan files | 14 | | backlog lines | 2,835 | | lessons lines | 455 | diff --git a/moondeck/ci/linux/postinst b/moondeck/ci/linux/postinst index 733baf14e..36b401fb7 100755 --- a/moondeck/ci/linux/postinst +++ b/moondeck/ci/linux/postinst @@ -14,10 +14,10 @@ if [ "$1" = configure ]; then if [ -e /etc/systemd/system/moonlight.service ]; then echo "moonlight: /etc/systemd/system/moonlight.service overrides the packaged service: remove it and run 'systemctl daemon-reload' (MIGRATING)" >&2 fi -fi -# Where systemd runs: a container build unpacks the package and runs none of this. -if [ -d /run/systemd/system ]; then - systemctl daemon-reload - systemctl enable moonlight.service - systemctl restart moonlight.service + # Where systemd runs: a container build unpacks the package and runs none of this. + if [ -d /run/systemd/system ]; then + systemctl daemon-reload + systemctl enable moonlight.service + systemctl restart moonlight.service + fi fi diff --git a/moonwalk/02-install.yaml b/moonwalk/02-install.yaml index 22f3916e6..82e9e6293 100644 --- a/moonwalk/02-install.yaml +++ b/moonwalk/02-install.yaml @@ -249,7 +249,7 @@ processes: | Teensy | In 1 MB of the program flash. | | Windows | `%LOCALAPPDATA%\MoonLight`: paste it into Explorer's address bar. Uninstalling leaves it. | | macOS | `~/Library/Application Support/MoonLight`. | - | Linux | `~/.local/share/MoonLight`, or under `$XDG_DATA_HOME` when it is set. | + | Linux | `/var/lib/moonlight/MoonLight` for the package's service; `~/.local/share/MoonLight`, or `$XDG_DATA_HOME`, run by hand. | | Docker | The `moonlight-data` volume; without it, every `docker rm` starts over. | - id: update section: Updating @@ -262,11 +262,11 @@ processes: A Teensy has no TLS, so the browser downloads the release's `.bin` and uploads it. Power lost right after **Rebooting…** leaves it dark until a USB install with the program button, which keeps the settings. - id: update-app section: Updating - say: Install the new build. + say: Update from the ⬆ badge. given: [{ input: platform, in: [Windows, macOS, Linux] }] - when: { manual: "The ⬆ badge in the top bar opens its release page." } + when: { manual: "It opens the Firmware card, or the release page for a program updated by hand." } detail: | - The badge lights when a newer stable release ships for your OS. A running program cannot replace its own executable, so updating is the download and the install again, in about a minute. Your settings stay. + It lights for a newer stable release; your settings stay. A program in a folder it can write installs it and restarts, as a board does; otherwise, download and install it again. - id: update-docker section: Updating say: Recreate the container from the new image. diff --git a/src/core/util/ProgramImage.h b/src/core/util/ProgramImage.h index 39f308f91..c343e902e 100644 --- a/src/core/util/ProgramImage.h +++ b/src/core/util/ProgramImage.h @@ -7,13 +7,15 @@ /// /// A desktop vets a new version of itself as a board vets an image (`FirmwareImage.h`), by the header its system's loader reads: /// -/// | System | Format | Magic | Processor field | -/// |--------|--------|-------|-----------------| -/// | Linux | ELF (64-bit) | `7F 'E' 'L' 'F'`, class 2 | `e_machine` at 18: 62 x86-64, 183 AArch64 | -/// | macOS | Mach-O (64-bit) | `CF FA ED FE` | `cputype` at 4: `0x01000007` x86-64, `0x0100000C` arm64 | -/// | Windows | PE | `'M' 'Z'`, then `"PE\0\0"` at the offset at `0x3C` | `Machine` after it: `0x8664` x64, `0xAA64` arm64 | +/// | System | Format | Magic | Processor field | Executable | +/// |--------|--------|-------|-----------------|------------| +/// | Linux | ELF (64-bit) | `7F 'E' 'L' 'F'`, class 2 | `e_machine` at 18: 62 x86-64, 183 AArch64 | `e_type` at 16: 2 `ET_EXEC`, or 3 `ET_DYN`, a PIE | +/// | macOS | Mach-O (64-bit) | `CF FA ED FE` | `cputype` at 4: `0x01000007` x86-64, `0x0100000C` arm64 | `filetype` at 12: 2 `MH_EXECUTE` | +/// | Windows | PE | `'M' 'Z'`, then `"PE\0\0"` at the offset at `0x3C` | `Machine` after it: `0x8664` x64, `0xAA64` arm64 | `Characteristics` 18 bytes on: `EXECUTABLE_IMAGE` set, `DLL` clear | /// -/// All three store their fields little-endian on these processors. +/// All three store their fields little-endian on these processors, and each header is read whole: 64 bytes for ELF, 32 for Mach-O, 24 from the PE signature. +/// +/// A shared library is `ET_DYN` as a PIE is, so ELF tells the two apart only by the loader it names; the processor and the digest still hold for it. /// /// A universal macOS binary is refused, the release shipping one processor per file. @@ -38,26 +40,28 @@ inline uint32_t le32(std::span b, size_t at) { /// What a header says: whether it is a program at all, the system that loads it, and the processor field it names. struct Header { bool program = false; ///< a known executable format at all + bool executable = false; ///< a program the loader starts, a library or an object file otherwise ProgramSystem system = ProgramSystem::Linux; ///< the system whose loader reads it bool x64 = false; ///< built for x86-64 bool arm64 = false; ///< built for arm64 }; inline Header elf(std::span b) { - if (b.size() < 20 || b[0] != 0x7F || b[1] != 'E' || b[2] != 'L' || b[3] != 'F' || b[4] != 2) return {}; - return {true, ProgramSystem::Linux, le16(b, 18) == 62, le16(b, 18) == 183}; + if (b.size() < 64 || b[0] != 0x7F || b[1] != 'E' || b[2] != 'L' || b[3] != 'F' || b[4] != 2) return {}; + return {true, le16(b, 16) == 2 || le16(b, 16) == 3, ProgramSystem::Linux, le16(b, 18) == 62, le16(b, 18) == 183}; } inline Header machO(std::span b) { - if (b.size() < 8 || le32(b, 0) != 0xFEEDFACFu) return {}; - return {true, ProgramSystem::MacOS, le32(b, 4) == 0x01000007u, le32(b, 4) == 0x0100000Cu}; + if (b.size() < 32 || le32(b, 0) != 0xFEEDFACFu) return {}; + return {true, le32(b, 12) == 2, ProgramSystem::MacOS, le32(b, 4) == 0x01000007u, le32(b, 4) == 0x0100000Cu}; } inline Header pe(std::span b) { if (b.size() < 0x40 || b[0] != 'M' || b[1] != 'Z') return {}; const uint32_t peAt = le32(b, 0x3C); - if (b.size() < static_cast(peAt) + 6 || le32(b, peAt) != 0x00004550u) return {}; - return {true, ProgramSystem::Windows, le16(b, peAt + 4) == 0x8664, le16(b, peAt + 4) == 0xAA64}; + if (b.size() < static_cast(peAt) + 24 || le32(b, peAt) != 0x00004550u) return {}; + const uint16_t traits = le16(b, peAt + 22); + return {true, (traits & 0x0002) && !(traits & 0x2000), ProgramSystem::Windows, le16(b, peAt + 4) == 0x8664, le16(b, peAt + 4) == 0xAA64}; } } // namespace program_image @@ -68,6 +72,7 @@ inline const char* programRejection(std::span head, ProgramSystem if (!h.program) h = program_image::pe(head); if (!h.program) return "not a program"; // a 404 page, an archive, a board's image if (h.system != system) return "a program for another system"; + if (!h.executable) return "a library or an object file"; return (cpu == ProgramCpu::Arm64 ? h.arm64 : h.x64) ? nullptr : "a program for another processor"; } diff --git a/src/platform/desktop/platform_desktop.cpp b/src/platform/desktop/platform_desktop.cpp index c99f6b08f..6f6d016af 100644 --- a/src/platform/desktop/platform_desktop.cpp +++ b/src/platform/desktop/platform_desktop.cpp @@ -665,11 +665,12 @@ struct FetchSink { size_t len = 0; bool tooBig = false; bool refused = false; ///< the stream turned a chunk down, which ends the transfer as failed + bool cut = false; ///< the transport broke off mid-answer DownloadChunkFn stream = nullptr; ///< a download's receiver, which takes the answer a chunk at a time void* user = nullptr; uint32_t* total = nullptr; ///< where a download learns the answer's size, once its headers name it - /// Whether the answer was cut short, by a body too big for the caller or a chunk the stream refused. - bool failed() const { return tooBig || refused; } + /// Whether the answer was cut short: a body too big for the caller, a chunk the stream refused, or a transport that broke off. + bool failed() const { return tooBig || refused || cut; } /// Take `n` more bytes, false once they would not fit with the terminator or the stream refuses them. bool take(const char* data, size_t n) { if (stream) { @@ -708,8 +709,11 @@ void readAnswer(HINTERNET request, DWORD status, FetchSink& sink) { if (sink.stream && status != 200u) return; // DWORD: an unsigned literal keeps MSVC's /W4 quiet char chunk[1024]; DWORD got = 0; - while (WinHttpReadData(request, chunk, sizeof(chunk), &got) && got > 0) - if (!sink.take(chunk, got)) break; + // A failed read is a broken transfer, apart from the zero-byte read that ends the answer. + for (;;) { + if (!WinHttpReadData(request, chunk, sizeof(chunk), &got)) { sink.cut = true; return; } + if (got == 0 || !sink.take(chunk, got)) return; + } } /// One request over WinHTTP into `sink`; the status, or 0. @@ -3562,6 +3566,9 @@ static bool spawnEncoderProcess(const char* const argv[], bool captureStdout = f return false; } ::signal(SIGPIPE, SIG_IGN); // a dead ffmpeg surfaces as EPIPE, not a signal + // Ours close on exec, so a restart in place hands the encoder its EOF rather than this run's pipes. + close_on_exec(fds[1]); + if (outFds[0] >= 0) close_on_exec(outFds[0]); encPid_ = pid; encStdin_ = fds[1]; encStdout_ = outFds[0]; diff --git a/src/platform/desktop/platform_desktop_update.cpp b/src/platform/desktop/platform_desktop_update.cpp index 3093f45a1..50e56a886 100644 --- a/src/platform/desktop/platform_desktop_update.cpp +++ b/src/platform/desktop/platform_desktop_update.cpp @@ -188,19 +188,21 @@ std::vector commandLine() { /// The fetch's own thread: the image downloaded into the writer, put in place, and the program restarted on it. void fetchTask(const std::string& url, const std::string& fallbackUrl, const std::string& sha256Hex, char* status, size_t statusLen, uint32_t* bytesRead, uint32_t* bytesTotal) { - for (const std::string& from : {url, fallbackUrl}) { - if (from.empty()) continue; + const std::string* const sources[] = {&url, &fallbackUrl}; + for (const std::string* from : sources) { + if (from->empty()) continue; OtaWrite w; if (!otaBegin(w, OtaTarget::App, 0, status, statusLen, bytesRead)) return; static_cast(w.impl)->sha256 = sha256Hex; - const int httpStatus = httpDownload(from.c_str(), 10 * 60 * 1000, + const int httpStatus = httpDownload(from->c_str(), 10 * 60 * 1000, [](void* user, const char* data, size_t n) { return otaWrite(*static_cast(user), reinterpret_cast(data), n); }, &w, bytesTotal); if (httpStatus != 200) { if (w.impl) abandon(w); - say(status, statusLen, "error: the download failed"); + // The update stays in flight while the other address is still to try, so no second update starts on MoonLight.next meanwhile. + say(status, statusLen, from == &url && !fallbackUrl.empty() ? "downloading" : "error: the download failed"); continue; } if (otaEnd(w, true)) { @@ -211,10 +213,8 @@ void fetchTask(const std::string& url, const std::string& fallbackUrl, const std } } -} // namespace - -fs::path programPath() { - if (!g_testProgram.empty()) return g_testProgram; +/// The running program's file as the system names it, read once at start: once a new program takes the name, Linux names the running one "... (deleted)". +fs::path executablePath() { std::error_code ec; #if defined(_WIN32) std::wstring path(32768, L'\0'); @@ -230,6 +230,12 @@ fs::path programPath() { return fs::read_symlink("/proc/self/exe", ec); #endif } +} // namespace + +fs::path programPath() { + static const fs::path installed = executablePath(); + return g_testProgram.empty() ? installed : g_testProgram; +} void setTestProgramPath(const fs::path& path) { g_testProgram = path; } @@ -252,9 +258,16 @@ bool putProgramInPlace(const fs::path& next, const fs::path& program, const char return refuse(why); } if (sha256Hex && *sha256Hex && !digestMatches(next, sha256Hex)) return refuse("error: the image does not match the release's digest"); - if (const char* failure = swapIn(next, program)) return refuse(failure); - // Pending until the new program has run a minute, which is when the service's rollback stops applying to it. - if (std::FILE* pending = openFile(withSuffix(program, ".pending"), "wb")) std::fclose(pending); + // Pending until the new program has run a minute, which is when the service's rollback stops applying to it; marked first, so no new program runs unmarked. + const fs::path pending = withSuffix(program, ".pending"); + std::FILE* marker = openFile(pending, "wb"); + if (!marker) return refuse("error: cannot mark the new program pending"); + std::fclose(marker); + if (const char* failure = swapIn(next, program)) { + std::error_code ignored; + fs::remove(pending, ignored); + return refuse(failure); + } return true; } @@ -317,6 +330,7 @@ bool otaEnd(OtaWrite& w, bool commit) { const bool closed = std::fclose(img->file) == 0; img->file = nullptr; if (!commit || !closed) { + if (commit) say(img->status, img->statusLen, "error: cannot write the new program"); abandon(w); return false; } diff --git a/src/ui/app.js b/src/ui/app.js index 7009e1382..3efde037e 100644 --- a/src/ui/app.js +++ b/src/ui/app.js @@ -1624,16 +1624,10 @@ function watchInstall(kind) { ui.status(`Installing ${kind.label.toLowerCase()}\u2026`); sawWork = true; } - // ANSWERING AGAIN AFTER SILENCE is the rebooting install's success: the image it - // wrote is the one now running. - if (kind.reboots && sawWork && silent > 0) { - ui.status(`${kind.label} installed`); - await fwSleep(1200); - location.reload(); - return; - } - // Back to rest with no error is the non-rebooting install's success. - if (!kind.reboots && sawWork && (text === "" || text === "idle")) { + // Back to rest with no error is the success: a fresh run starts at rest, so a rebooting + // install that comes back between two probes shows it too. ANSWERING AGAIN AFTER SILENCE + // is a rebooting install's success as well: the image it wrote is the one now running. + if (sawWork && (text === "" || text === "idle" || (kind.reboots && silent > 0))) { ui.status(`${kind.label} installed`); await fwSleep(1200); location.reload(); diff --git a/src/ui/install-picker.js b/src/ui/install-picker.js index f23db8f3f..5dfcbac72 100644 --- a/src/ui/install-picker.js +++ b/src/ui/install-picker.js @@ -285,6 +285,14 @@ function mergeFirmwares(published, extras) { // "This computer" and on the Firmware card of a desktop its host or its user updates. export const DESKTOP_DOWNLOAD = "desktop-download"; +// Whether `f` is the kind on offer: a download in download mode, a board's firmware or a desktop's own program otherwise. +// A desktop's archive and its program share one firmware key, so the kind tells them apart, in the list and at the install alike. +const ofOfferedKind = (f, ownFirmwareKey) => !!f.isDesktop === (ownFirmwareKey === DESKTOP_DOWNLOAD); + +// The entry an install acts on: the picked firmware, of the kind on offer. +export const pickedEntry = (firmwares, firmware, ownFirmwareKey) => + (firmwares || []).find(f => f.firmware === firmware && ofOfferedKind(f, ownFirmwareKey)); + // A desktop's own program, firmware-desktop---v.bin, which a running desktop installs over the air. const isDesktopProgram = (firmware) => /^desktop-(?!download$)/.test(firmware || ""); @@ -578,7 +586,7 @@ function render(state) { // row decides which kind is on offer. const wantDesktop = state.ownFirmwareKey === DESKTOP_DOWNLOAD; let compatible = (r.firmwares || []) - .filter(f => !!f.isDesktop === wantDesktop) + .filter(f => ofOfferedKind(f, state.ownFirmwareKey)) // A MoonBase key is a CHIP ("esp32s3"), not a firmware variant, so the variant rule // (which strips -eth and compares) does not apply: an exact chip match is the whole // question, and offering another chip's image is what the device's own header check @@ -782,7 +790,7 @@ function render(state) { // enablement shouldn't let a flash through). The dropdown stays the place // to pick a port. if (state.blocker && state.blocker()) { applyInstallEnabled(); return; } - const entry = (r.firmwares || []).find(f => f.firmware === state.firmware); + const entry = pickedEntry(r.firmwares, state.firmware, state.ownFirmwareKey); if (!entry) return; // Mismatch guard: if Detect ran and the user then overrode the device to // one of a different chip family, confirm before flashing the wrong diff --git a/test/js/installer-desktop-download.test.mjs b/test/js/installer-desktop-download.test.mjs index e08d28d9d..4c2e93864 100644 --- a/test/js/installer-desktop-download.test.mjs +++ b/test/js/installer-desktop-download.test.mjs @@ -14,7 +14,7 @@ import { readFileSync } from "node:fs"; import { dirname, join } from "node:path"; import { fileURLToPath } from "node:url"; -import { parseFirmwaresFromAssets, isCompatible, DESKTOP_DOWNLOAD } from "../../src/ui/install-picker.js"; +import { parseFirmwaresFromAssets, isCompatible, pickedEntry, DESKTOP_DOWNLOAD } from "../../src/ui/install-picker.js"; const ROOT = join(dirname(fileURLToPath(import.meta.url)), "..", ".."); const asset = (name) => ({ name, browser_download_url: `https://gh/${name}` }); @@ -159,3 +159,14 @@ test("a release with only an archive still offers it, for every platform it has" assert.match(got.find(f => f.firmware === "desktop-macos-arm64").binaryUrl, /\.tar\.gz$/); assert.match(got.find(f => f.firmware === "desktop-windows-x64").binaryUrl, /\.zip$/); }); + +test("installing a desktop's own program installs the program, and downloading it downloads the package, both in one release", () => { + // The package and the program share the key desktop-linux-x64; the install takes the one of the kind on offer. + const firmwares = parseFirmwaresFromAssets([...RELEASE, asset("firmware-desktop-linux-x64-v3.0.0.bin")]); + const program = pickedEntry(firmwares, "desktop-linux-x64", "desktop-linux-x64"); + assert.equal(program.binaryUrl, "https://gh/firmware-desktop-linux-x64-v3.0.0.bin"); + assert.ok(!program.isDesktop); + const download = pickedEntry(firmwares, "desktop-linux-x64", DESKTOP_DOWNLOAD); + assert.ok(download.isDesktop); + assert.ok(download.assets.some(a => a.name.endsWith(".deb"))); +}); diff --git a/test/unit/core/unit_HttpServerModule_routes.cpp b/test/unit/core/unit_HttpServerModule_routes.cpp index 373ea4335..0e84c843b 100644 --- a/test/unit/core/unit_HttpServerModule_routes.cpp +++ b/test/unit/core/unit_HttpServerModule_routes.cpp @@ -138,8 +138,9 @@ TEST_CASE("a firmware upload without a Content-Length is refused with 411, as ev http.release(); } -// A name past what a module holds is refused on every route that names one, rather than cut short into a name that could match another module. -TEST_CASE("an image URL longer than the MoonBase handoff takes is refused with 400 rather than fetched cut short") { +// The route reads the URL on a device that updates itself; a container or a read-only build folder answers 501 first. +TEST_CASE("an image URL longer than the MoonBase handoff takes is refused with 400 rather than fetched cut short" + * doctest::skip(mm::platform::updateMode() != mm::platform::UpdateMode::Self)) { mm::HttpServerModule http; http.port = kTestPort; http.setup(); @@ -155,6 +156,7 @@ TEST_CASE("an image URL longer than the MoonBase handoff takes is refused with 4 http.release(); } +// A name past what a module holds is refused on every route that names one, rather than cut short into a name that could match another module. TEST_CASE("a module route whose name does not fit is refused with 400 on move, replace, delete and read alike") { RackServer server; mm::HttpServerModule& http = server.http; diff --git a/test/unit/core/unit_ProgramImage.cpp b/test/unit/core/unit_ProgramImage.cpp index 2ab7988c0..d1f981c65 100644 --- a/test/unit/core/unit_ProgramImage.cpp +++ b/test/unit/core/unit_ProgramImage.cpp @@ -2,7 +2,7 @@ /// Pins what a desktop program's first bytes say about it, the check a desktop runs before a new version replaces it, as a board checks its image's chip. /// -/// An ELF, Mach-O or PE header for the right system and processor passes, another one is refused with its reason, and a web page or an archive is no program. +/// An ELF, Mach-O or PE program for the right system and processor passes; another one, a library or an object file is refused with its reason, and a web page, an archive or a cut header is no program. #include "doctest.h" #include "core/util/ProgramImage.h" @@ -18,26 +18,29 @@ namespace { void putLe16(std::vector& b, size_t at, uint16_t v) { b[at] = static_cast(v); b[at + 1] = static_cast(v >> 8); } void putLe32(std::vector& b, size_t at, uint32_t v) { for (int i = 0; i < 4; i++) b[at + i] = static_cast(v >> (8 * i)); } -std::vector elf(uint16_t machine) { +std::vector elf(uint16_t machine, uint16_t type = 3) { // 3: a PIE, as the release builds std::vector b(64, 0); b[0] = 0x7F; b[1] = 'E'; b[2] = 'L'; b[3] = 'F'; b[4] = 2; + putLe16(b, 16, type); putLe16(b, 18, machine); return b; } -std::vector machO(uint32_t cpu) { +std::vector machO(uint32_t cpu, uint32_t filetype = 2) { // 2: MH_EXECUTE std::vector b(32, 0); putLe32(b, 0, 0xFEEDFACFu); putLe32(b, 4, cpu); + putLe32(b, 12, filetype); return b; } -std::vector pe(uint16_t machine) { +std::vector pe(uint16_t machine, uint16_t characteristics = 0x0022) { // EXECUTABLE_IMAGE | LARGE_ADDRESS_AWARE std::vector b(0x100, 0); b[0] = 'M'; b[1] = 'Z'; putLe32(b, 0x3C, 0x80); putLe32(b, 0x80, 0x00004550u); putLe16(b, 0x84, machine); + putLe16(b, 0x96, characteristics); return b; } @@ -59,6 +62,14 @@ TEST_CASE("a program for another system or another processor is refused with tha CHECK(std::string(programRejection(machO(0x01000007u), ProgramSystem::MacOS, ProgramCpu::Arm64)) == "a program for another processor"); } +TEST_CASE("a fixed-address ELF program passes as a PIE does, and a library or an object file is refused, in each format") { + CHECK(programRejection(elf(62, 2), ProgramSystem::Linux, ProgramCpu::X64) == nullptr); // ET_EXEC + const char* const why = "a library or an object file"; + CHECK(std::string(programRejection(elf(62, 1), ProgramSystem::Linux, ProgramCpu::X64)) == why); // ET_REL + CHECK(std::string(programRejection(machO(0x0100000Cu, 6), ProgramSystem::MacOS, ProgramCpu::Arm64)) == why); // MH_DYLIB + CHECK(std::string(programRejection(pe(0x8664, 0x2022), ProgramSystem::Windows, ProgramCpu::X64)) == why); // a DLL +} + TEST_CASE("a web page, an archive or a truncated header is no program") { const std::string page = "404"; CHECK(std::string(programRejection(std::vector(page.begin(), page.end()), ProgramSystem::Linux, ProgramCpu::X64)) == "not a program"); @@ -67,4 +78,7 @@ TEST_CASE("a web page, an archive or a truncated header is no program") { shortPe.resize(0x82); // the PE signature's offset points past the end CHECK(std::string(programRejection(shortPe, ProgramSystem::Windows, ProgramCpu::X64)) == "not a program"); CHECK(std::string(programRejection({}, ProgramSystem::Windows, ProgramCpu::X64)) == "not a program"); + std::vector shortElf = elf(62); + shortElf.resize(20); // magic, class and machine, the rest of the header missing + CHECK(std::string(programRejection(shortElf, ProgramSystem::Linux, ProgramCpu::X64)) == "not a program"); } diff --git a/test/unit/core/unit_platform_update.cpp b/test/unit/core/unit_platform_update.cpp index 0fe0098d4..43b9df6e9 100644 --- a/test/unit/core/unit_platform_update.cpp +++ b/test/unit/core/unit_platform_update.cpp @@ -13,6 +13,7 @@ #include #include +#include #include #include @@ -25,7 +26,7 @@ namespace fs = std::filesystem; namespace { -// FILE rather than a stream: a stream's object takes more stack than the frame limit allows. +// Read through FILE: a stream's object takes more stack than the frame limit allows. std::vector readBytes(const fs::path& path) { std::vector bytes; std::FILE* file = std::fopen(path.string().c_str(), "rb"); @@ -36,11 +37,11 @@ std::vector readBytes(const fs::path& path) { return bytes; } +// Written through a stream, whose file takes the process's umask, as the program's own files do. void writeBytes(const fs::path& path, const std::vector& bytes) { - std::FILE* file = std::fopen(path.string().c_str(), "wb"); - REQUIRE(file != nullptr); - std::fwrite(bytes.data(), 1, bytes.size(), file); - std::fclose(file); + std::ofstream file(path, std::ios::binary | std::ios::trunc); + REQUIRE(file.good()); + file.write(reinterpret_cast(bytes.data()), static_cast(bytes.size())); } void writeText(const fs::path& path, const std::string& text) { writeBytes(path, std::vector(text.begin(), text.end())); } @@ -96,9 +97,12 @@ struct Scratch { } }; +/// A container's program is its host's to replace, so the self-update cases run on a native host only. +const auto kSelfUpdating = doctest::skip(platform::inContainer()); + } // namespace -TEST_CASE("a native program in a folder it can write updates itself, one in a folder it cannot write is updated by hand") { +TEST_CASE("a native program in a folder it can write updates itself, one in a folder it cannot write is updated by hand" * kSelfUpdating) { Scratch s("mode"); CHECK(platform::updateMode() == platform::UpdateMode::Self); #ifndef _WIN32 @@ -113,7 +117,7 @@ TEST_CASE("a native program in a folder it can write updates itself, one in a fo CHECK(std::string(platform::updateModeName(platform::UpdateMode::Manual)) == "manual"); } -TEST_CASE("an image written through the image writer replaces the program and keeps the old one as .prev, pending until it has run") { +TEST_CASE("an image written through the image writer replaces the program and keeps the old one as .prev, pending until it has run" * kSelfUpdating) { Scratch s("install"); REQUIRE(s.install(s.realProgram)); CHECK(readBytes(s.program) == s.realProgram); @@ -122,7 +126,7 @@ TEST_CASE("an image written through the image writer replaces the program and ke CHECK_FALSE(fs::exists(s.dir / "MoonLight.next")); } -TEST_CASE("an image that is no program for this system leaves the program as it was") { +TEST_CASE("an image that is no program for this system leaves the program as it was" * kSelfUpdating) { Scratch s("refused"); const std::string page = "404 Not Found"; CHECK_FALSE(s.install(std::vector(page.begin(), page.end()))); @@ -131,14 +135,14 @@ TEST_CASE("an image that is no program for this system leaves the program as it CHECK_FALSE(fs::exists(s.dir / "MoonLight.next")); } -TEST_CASE("an abandoned image leaves the program as it was") { +TEST_CASE("an abandoned image leaves the program as it was" * kSelfUpdating) { Scratch s("abandoned"); CHECK_FALSE(s.install(s.realProgram, false)); CHECK(readText(s.program) == "old program"); CHECK_FALSE(fs::exists(s.dir / "MoonLight.next")); } -TEST_CASE("an image missing the release's digest leaves the program as it was, and one matching it in either case is installed") { +TEST_CASE("an image missing the release's digest leaves the program as it was, and one matching it in either case is installed" * kSelfUpdating) { Scratch s("digest"); const fs::path next = s.dir / "MoonLight.next"; writeBytes(next, s.realProgram); From 4bc8eab67614b79cf454e01faa217df0545881fe Mon Sep 17 00:00:00 2001 From: ewowi Date: Fri, 13 Nov 2026 12:19:38 +0100 Subject: [PATCH 3/5] Every desktop build installs its own program in CI before it ships Each desktop job in the release workflow now starts the program it built, installs that program into it through its own Firmware upload route and checks it comes back on it, so the self-update is tested on Windows, macOS and Linux on every pull request. The Windows build carries its real version, and the Linux service's rollback is described as systemd runs it. KPI: not run, no firmware or desktop source changed (a CI script, the workflow and docs). **Scripts/MoonDeck** - release_stage.py smoke-update: copies the program into a folder of its own, waits for update: self, uploads the program to itself, and passes when a new run answers from a lower uptime with .prev and .pending beside it and no .next left behind; a Windows restart is a new process, stopped by name. Tested on this Mac against the desktop build: upload 200, uptime 5 s -> 0 s, in about 8 s. **Docs/CI** - release.yml: the smoke-update step in the macOS, Windows, Linux x64 and Linux arm64 jobs. The Windows packaging step runs under bash, since PowerShell read "$VERSION" as its own empty variable and packaged library.json's bare 7.0.0-dev, which the latest release's Windows files also carry. - moonlight.service and installing-on-linux: the rollback starts on a failed start of a pending program, as OnFailure= fires on every failure on the Pi's systemd; StartLimit stops a program that keeps failing after that. - MoonDeck.md names smoke-update. The plan records step 1's tests: this Mac, .204's service (self-update in place, rollback from /bin/false, reinstall, HDMI under the service user) and .156's container (update: host, both install routes 501, restart in place); Windows verified by smoke-update. **Tests** - Release-stage tests 19 passed; prose, docgen (1747) and the project mirror pass; smoke_update and its helpers stay under check_code's complexity and length limits. **Reviews** - CodeRabbit's review of the previous commit: no actionable comments. - collect_kpi: SKIP, no src/ change. Desktop build and unit tests: SKIP, no C++ or JS change. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/release.yml | 17 +++++ docs/reference/installing-on-linux.md | 2 +- ...9 - Devices, groups and the network map.md | 9 +-- moondeck/MoonDeck.md | 2 +- moondeck/ci/linux/moonlight.service | 3 +- moondeck/ci/release_stage.py | 72 +++++++++++++++++++ 6 files changed, 98 insertions(+), 7 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index d3708fe48..e928ca246 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -250,6 +250,10 @@ jobs: env: VERSION: ${{ steps.ver.outputs.version }} run: uv run moondeck/ci/package_desktop.py --version "$VERSION" + # The program installs itself through its own Firmware route and restarts on it, on the system it is built for. + - name: Install its own program and restart on it + shell: bash + run: uv run moondeck/ci/release_stage.py smoke-update --program dist/firmware-desktop-*.bin - uses: actions/upload-artifact@v4 with: name: desktop-macos @@ -314,9 +318,14 @@ jobs: } Write-Host "NSIS present: $found" - name: Build + package Windows x64 + shell: bash # PowerShell would read "$VERSION" as its own, empty variable and package library.json's bare version env: VERSION: ${{ steps.ver.outputs.version }} run: uv run moondeck/ci/package_desktop.py --version "$VERSION" + # The program installs itself through its own Firmware route and restarts on it, on the system it is built for. + - name: Install its own program and restart on it + shell: bash + run: uv run moondeck/ci/release_stage.py smoke-update --program dist/firmware-desktop-*.bin - uses: actions/upload-artifact@v4 with: name: desktop-windows @@ -376,6 +385,10 @@ jobs: env: VERSION: ${{ steps.ver.outputs.version }} run: uv run moondeck/ci/package_desktop.py --version "$VERSION" + # The program installs itself through its own Firmware route and restarts on it, on the system it is built for. + - name: Install its own program and restart on it + shell: bash + run: uv run moondeck/ci/release_stage.py smoke-update --program dist/firmware-desktop-*.bin - name: Check the glibc floor run: uv run moondeck/ci/check_glibc_floor.py dist/moonlight_*_amd64.deb - uses: actions/upload-artifact@v4 @@ -435,6 +448,10 @@ jobs: env: VERSION: ${{ steps.ver.outputs.version }} run: uv run moondeck/ci/package_desktop.py --version "$VERSION" + # The program installs itself through its own Firmware route and restarts on it, on the system it is built for. + - name: Install its own program and restart on it + shell: bash + run: uv run moondeck/ci/release_stage.py smoke-update --program dist/firmware-desktop-*.bin - name: Check the glibc floor run: uv run moondeck/ci/check_glibc_floor.py dist/moonlight_*_arm64.deb - uses: actions/upload-artifact@v4 diff --git a/docs/reference/installing-on-linux.md b/docs/reference/installing-on-linux.md index 539a78b65..484ecf850 100644 --- a/docs/reference/installing-on-linux.md +++ b/docs/reference/installing-on-linux.md @@ -27,7 +27,7 @@ The package installs MoonLight as `moonlight.service` and starts it, so the mach **Updating, as a board updates.** The service owns the folder its program runs from, so it installs a new version of itself through the same routes, Firmware card and map as a board, from the release's `firmware-desktop-linux--v….bin`. It checks the image is a program for this processor and, from the map, against the SHA-256 GitHub publishes for it; it keeps the running program as `MoonLight.prev`, puts the new one in its place and restarts on it in place, the same process systemd started. Its `/api/system` says `update: self`. A package-managed program that updates itself is uncommon on Linux, where the textbook is an APT repository; that needs root to update from the map, and the network-facing service stays unprivileged, so the service owns a copy of the program the package seeds, as Syncthing does. -**Rolling back.** A new program stays pending, marked by `MoonLight.pending`, until it has run a minute. A pending program that fails three starts within two minutes gives way to `MoonLight.prev`, through `moonlight-rollback.service`; a confirmed one stays in place through any number of restarts. An `apt install` is the administrator's choice: it puts the package's program in place, older or newer, and drops `MoonLight.prev`. +**Rolling back.** A new program stays pending, marked by `MoonLight.pending`, until it has run a minute. A pending program whose start fails gives way to `MoonLight.prev`, through `moonlight-rollback.service`; a confirmed one stays in place through its restarts. An `apt install` is the administrator's choice: it puts the package's program in place, older or newer, and drops `MoonLight.prev`. **Running it by hand**, to see its log in a terminal: `sudo systemctl stop moonlight`, then `/usr/lib/moonlight/MoonLight`, which keeps its settings in your own `~/.local/share/MoonLight/`. That copy sits in a folder only root writes, so it is updated by hand; the tarball unpacked in a folder of your own updates itself, restarting in its terminal window. diff --git a/docs/work/Plan-20261109 - Devices, groups and the network map.md b/docs/work/Plan-20261109 - Devices, groups and the network map.md index d3e7811d5..e1719a6e8 100644 --- a/docs/work/Plan-20261109 - Devices, groups and the network map.md +++ b/docs/work/Plan-20261109 - Devices, groups and the network map.md @@ -168,7 +168,7 @@ Planned by the product owner (2026-11-10) after step 5 and before step 6, each s - It needs a hardware test: mDNS queries were ended by a pool exhaustion on the ESP32, so the passive listen is a hypothesis until a board runs it. - **Verified by:** a Hue bridge nobody configured on the bench map. -#### Every desktop updates itself through the Firmware module 🧪 (test: an install on this Mac, .204 and a Windows PC from their own Firmware card; the CI build on Linux and Windows) +#### Every desktop updates itself through the Firmware module 🧪 (test: Windows, by CI's `smoke-update` on the next push) Asked by the product owner (2026-11-12): one way to update any MoonLight device, in two steps, this one and the map using it. A desktop prepares the new version beside the running one and a restart activates it, as a board writes its other slot and reboots into it, through the same routes, the same Firmware card and the same asset naming. The line runs between a native install, which updates itself on every OS, and a container, which its host replaces. Settled with the product owner: the Linux package runs a service enabled on install, rollback by systemd, settings moved by hand, the digest from the browser. @@ -177,13 +177,14 @@ Asked by the product owner (2026-11-12): one way to update any MoonLight device, - **The restart activates it.** `platform::reboot()` starts the program again: on macOS and Linux the new program takes over the process in place (`execv`), same terminal window, same process to systemd or a container runtime, so a reboot is never a start their restart counting sees, with every socket marked close-on-exec where it is made; on Windows the new `.exe` starts in the same console and the old one exits. The reboot button shows on every device. - **The update mode.** `/api/system`'s `update`, and the Firmware card's: `self` for a native install whose program sits in a folder it can write, `host` for a container, `manual` for a program in a folder it cannot write, such as the package's seed under `/usr/lib`. - **The Linux package runs a service, and the service owns its program.** Bespoke, with its reason: the textbook Linux update is an APT repository, which needs root to update from the map, and the network-facing service never holds root; a self-owned copy beside a package-tracked seed is how Syncthing, a network daemon, replaces its own binary. New for MoonLight and uncommon on Linux, where the package manager is the norm. The `.deb` ships the program as a seed at `/usr/lib/moonlight/MoonLight`, a `moonlight` system user in the `video` group, and `moonlight.service`, which its post-install script enables and starts the way debhelper's snippets do, skipping where no systemd runs. The unit runs as that user with `StateDirectory=moonlight`, settings in `/var/lib/moonlight` (`XDG_DATA_HOME`), and `Restart=always`. The post-install script copies the seed to `/var/lib/moonlight/bin/MoonLight`, owned by the service user, on every install: an `apt install` is the administrator's explicit choice, so it wins over a self-update, older or newer, and drops `.prev`. The menu entry opens `http://localhost:8080/` instead of starting a second copy on the same port. No sudo rule and no Docker socket, so the network-facing service never holds root. -- **Rollback is systemd's, for the service, by boot assessment.** The swap leaves `MoonLight.pending` beside the program, and the new program deletes it once it has stayed up `kBootStableMs`, the minute a board's boot record uses: systemd's Automatic Boot Assessment, for one program. `StartLimitBurst=3` within `StartLimitIntervalSec=120` and `OnFailure=moonlight-rollback.service`: a pending program that fails three starts in two minutes, a crash before `main` included, is replaced by `.prev` and the service started again; a confirmed one is left alone, so restarts of a good program never undo it. Elsewhere `.prev` stays beside the program, to rename back by hand. +- **Rollback is systemd's, for the service, by boot assessment.** The swap leaves `MoonLight.pending` beside the program, and the new program deletes it once it has stayed up `kBootStableMs`, the minute a board's boot record uses: systemd's Automatic Boot Assessment, for one program. `OnFailure=moonlight-rollback.service`, which systemd starts on every failed start: a pending program that fails, a crash before `main` included, is replaced by `.prev` and the service started again; a confirmed one is left alone, so restarts of a good program never undo it. `StartLimitBurst=3` within `StartLimitIntervalSec=120` stops a program that keeps failing after that. Elsewhere `.prev` stays beside the program, to rename back by hand. - **The Firmware card on a desktop** installs as a board's does, by URL from its picker or by upload, when the update mode is `self`; otherwise it offers the download. The shared install picker tells a desktop's own image (`firmware-desktop-…bin`) from the installer's download of the same name. - **A hypothesis to test first:** a macOS bundle whose inner binary was replaced still launches, Terminal running the binary itself rather than the sealed bundle. -- **Built (2026-11-13).** The image writer, the swap and the restart in [platform_desktop_update.cpp](../../src/platform/desktop/platform_desktop_update.cpp), the header vetting in [ProgramImage.h](../../src/core/util/ProgramImage.h), the download through the one HTTP client (`httpDownload`), the firmware name from CMake, the sockets closed on exec, `hasOta` gone, the bare programs staged by `package_desktop.py`, the Firmware card installing on `self` and offering the download otherwise, the picker telling a desktop's program from its download, the live scenario leaving a desktop to restart itself. Tests: `unit_platform_update.cpp`, `unit_ProgramImage.cpp`, `ui-update-asset.test.mjs`, `test_package_deb.py`. Verified on this Mac: `firmware: desktop-macos-arm64`, `update: self`, and `/api/reboot` restarting in place, same process, uptime from zero. Linux and Windows compile in CI only so far. +- **Built (2026-11-13).** The image writer, the swap and the restart in [platform_desktop_update.cpp](../../src/platform/desktop/platform_desktop_update.cpp), the header vetting in [ProgramImage.h](../../src/core/util/ProgramImage.h), the download through the one HTTP client (`httpDownload`), the firmware name from CMake, the sockets closed on exec, `hasOta` gone, the bare programs staged by `package_desktop.py`, the Firmware card installing on `self` and offering the download otherwise, the picker telling a desktop's program from its download, the live scenario leaving a desktop to restart itself. Tests: `unit_platform_update.cpp`, `unit_ProgramImage.cpp`, `ui-update-asset.test.mjs`, `test_package_deb.py`. CodeRabbit's 15 findings and a CodeQL alert fixed in the PR's second commit, among them the Linux restart, which read `/proc/self/exe` after the swap and found the old program "(deleted)". - **Removes:** the archive path, the reboot button hidden on desktops, `unknown` as a desktop's firmware name, the Firmware card's desktop download in place of an install, MoonWalk's hand-written unit steps. **Fixed on the way:** the reboot button hidden for a container (`^docker$` never matched `docker-arm64`). - **Costs** a MIGRATING entry: the hand-written unit goes, the program moves from `/usr/bin` to `/usr/lib/moonlight`, and settings move from the old user's home to `/var/lib/moonlight` with one copy command. The Dockerfile copies the program from the new seed path. Unblocks step 9 for a group that mixes boards and desktops. -- **Verified by:** .204's package updated from its own Firmware card and rolled back from a program made to fail; this Mac and a Windows PC each updated from their own Firmware card, by URL and by upload, back in the same window on the new version; .156's container saying its host updates it. +- **Tested (2026-11-13), by the product owner and the agent.** This Mac: an upload installs and restarts in place, same PID, `.pending` cleared after a minute; a board image refused as no program; a URL with a wrong digest refused, with the right one installed and restarted. .204, the Raspberry Pi 5, from the PR's CI `.deb`: the service runs as `moonlight` holding `CAP_NET_RAW` alone, `update: self`; an upload restarts it in place, same PID, no start counted; `/bin/false` installed as an update rolls back to `.prev` within a second; an `apt install` puts the package's program back and drops `.prev`; HDMI shows the effect under the service user. .156, the NanoPi: the PR's program in its container image says `update: host`, refuses both install routes with 501 and restarts in place. The old hand-written unit and a source-build drop-in had to go first, which the MIGRATING entry covers for the unit. CI on the PR: every desktop compiles, Linux and Windows included. +- **Verified by:** .204's package updated from its own Firmware card and rolled back from a program made to fail; this Mac updated from its own Firmware card, by URL and by upload; Windows by CI's `smoke-update`, which installs a program through its own upload route and checks it restarts with `.prev` beside it, on each runner that builds a desktop; .156's container saying its host updates it. #### The map updates a desktop as it updates a board ⏳ (after the step above) diff --git a/moondeck/MoonDeck.md b/moondeck/MoonDeck.md index 05acb8a7e..4111fcd7c 100644 --- a/moondeck/MoonDeck.md +++ b/moondeck/MoonDeck.md @@ -182,7 +182,7 @@ Runs `./build//test/mm_tests -s` (doctest with all test cases shown) — s ### release_stage -The release's build and staging steps, one subcommand each, so `release.yml` calls a script for each and a release's assets stage locally too: `tag` and `version` for the release name, `firmwares` for the build matrix, `stage-esp32` and `stage-teensy` for the binaries, `gather`, `extras` and `manifests` for the release, `container-context` and `smoke-images` for the container, `pages-site` and `merge-docs` for the site. The upload, the `latest` tag, the registry push and Pages stay GitHub's own. The process: [CI on a release tag](https://moonmodules.org/MoonLight/moonwalk.html#ci-on-release). +The release's build and staging steps, one subcommand each, so `release.yml` calls a script for each and a release's assets stage locally too: `tag` and `version` for the release name, `firmwares` for the build matrix, `stage-esp32` and `stage-teensy` for the binaries, `gather`, `extras` and `manifests` for the release, `container-context` and `smoke-images` for the container, `smoke-update` for a desktop program installing itself on each runner that builds one, `pages-site` and `merge-docs` for the site. The upload, the `latest` tag, the registry push and Pages stay GitHub's own. The process: [CI on a release tag](https://moonmodules.org/MoonLight/moonwalk.html#ci-on-release). ```bash uv run moondeck/ci/release_stage.py tag --input-tag v7.1.0 --ref refs/tags/v7.1.0 --ref-name v7.1.0 diff --git a/moondeck/ci/linux/moonlight.service b/moondeck/ci/linux/moonlight.service index 0c40acf47..f6b9eb606 100644 --- a/moondeck/ci/linux/moonlight.service +++ b/moondeck/ci/linux/moonlight.service @@ -5,7 +5,8 @@ Description=MoonLight Documentation=https://moonmodules.org/MoonLight/reference/installing-on-linux/ After=network-online.target Wants=network-online.target -# Three starts in two minutes end in the failed state, which starts the rollback: a new program that keeps failing before it has run a minute gives way to the one before it. +# A failed start starts the rollback, which acts on a pending program only: one that fails before it has run a minute gives way to the one before it. +# Three starts in two minutes stop a program that keeps failing after that. StartLimitIntervalSec=120 StartLimitBurst=3 OnFailure=moonlight-rollback.service diff --git a/moondeck/ci/release_stage.py b/moondeck/ci/release_stage.py index 715751509..1439fab7f 100644 --- a/moondeck/ci/release_stage.py +++ b/moondeck/ci/release_stage.py @@ -19,6 +19,7 @@ image-name the lowercase container image path container-context the Docker build context holding the release .debs smoke-images run each container image before anything is published + smoke-update a desktop program installs itself through its Firmware route and restarts on it pages-site the installer and its assets, staged for Pages merge-docs the rendered docs site, copied into the Pages root beside the installer @@ -32,6 +33,7 @@ import shutil import subprocess import sys +import tempfile import time import urllib.request from pathlib import Path @@ -260,6 +262,72 @@ def smoke_images() -> None: print("the arm64 image runs") +def system_info(base: str) -> dict | None: + """The program's /api/system, or None while it does not answer.""" + try: + with urllib.request.urlopen(f"{base}/api/system", timeout=3) as r: + return json.loads(r.read()) + except (OSError, ValueError): + return None + + +def wait_for_system(base: str, accept, seconds: int = 60) -> dict | None: + """The first /api/system answer `accept` takes within `seconds`, or None.""" + for _ in range(seconds * 2): + info = system_info(base) + if info and accept(info): + return info + time.sleep(0.5) + return None + + +def check_beside(work: Path, program: str) -> None: + """The old program kept as .prev, the new one marked .pending, and no image left half-written.""" + files = sorted(p.name for p in work.iterdir() if p.is_file()) + print(f"beside the program: {files}") + missing = [kept for kept in (program + ".prev", program + ".pending") if kept not in files] + if missing or "MoonLight.next" in files: + raise SystemExit(f"beside the program: {', '.join(missing) or 'MoonLight.next'} {'missing' if missing else 'left behind'}") + + +def smoke_update(program: Path) -> None: + """A desktop installs a program through POST /api/firmware/upload as a board installs an image, and restarts on it: the new run answers from + a fresh uptime, the old program waits beside it as .prev and the new one is marked .pending. The program is its own image, a copy in a + folder of its own standing in for an installed one, so this runs on each runner the release builds a desktop on.""" + port = 18081 + base = f"http://127.0.0.1:{port}" + work = Path(tempfile.mkdtemp(prefix="mm-smoke-update-")) + installed = work / ("MoonLight.exe" if os.name == "nt" else "MoonLight") + shutil.copy2(program, installed) + log_path = work / "run.log" + with log_path.open("wb") as log: + run = subprocess.Popen([str(installed), "--port", str(port), "--no-browser"], stdout=log, stderr=subprocess.STDOUT, cwd=work, + env={**os.environ, "MM_DATA_DIR": str(work / "data")}) + try: + # Up a few seconds first, so the new run's uptime reads lower than the old one's. + before = wait_for_system(base, lambda info: info.get("uptime", 0) >= 5) + if not before or before.get("update") != "self": + raise SystemExit(f"the program did not answer as one that updates itself: {before}") + upload = urllib.request.Request(f"{base}/api/firmware/upload", data=installed.read_bytes(), method="POST", + headers={"Content-Type": "application/octet-stream"}) + with urllib.request.urlopen(upload, timeout=120) as r: + print(f"upload: {r.status} {r.read().decode(errors='replace')}") + after = wait_for_system(base, lambda info: info.get("uptime", 0) < before["uptime"]) + if not after: + raise SystemExit("no new run answered within 60 s") + print(f"restarted: uptime {before['uptime']} s -> {after['uptime']} s") + check_beside(work, installed.name) + print("the program installed itself and restarted on it") + finally: + # A POSIX restart is the same process, exec'd in place; a Windows restart is a new process, found by its name. + if os.name == "nt": + subprocess.run(["taskkill", "/F", "/IM", installed.name], capture_output=True) + run.kill() + run.wait() + print(tail(log_path.read_text(encoding="utf-8", errors="replace"))) + shutil.rmtree(work, ignore_errors=True) + + def pages_site() -> None: """The installer page, the picker modules and the board images a deviceModels.json entry references, under pages/install/.""" install = ROOT / "pages" / "install" @@ -327,6 +395,8 @@ def main() -> int: image.add_argument("--repo", required=True, help="owner/name") sub.add_parser("container-context", help="Write ctx/ with the .debs and the release Dockerfile.") sub.add_parser("smoke-images", help="Run the amd64 and arm64 smoke images.") + smoke = sub.add_parser("smoke-update", help="Install a desktop program through its own Firmware route and check it restarts on it.") + smoke.add_argument("--program", required=True, type=Path, help="dist/firmware-desktop---v.bin") sub.add_parser("pages-site", help="Stage the installer under pages/install/.") docs = sub.add_parser("merge-docs", help="Copy the rendered docs site into pages/.") docs.add_argument("--site-dir", required=True, type=Path) @@ -356,6 +426,8 @@ def main() -> int: container_context() elif args.command == "smoke-images": smoke_images() + elif args.command == "smoke-update": + smoke_update(args.program) elif args.command == "pages-site": pages_site() else: From 61309951d94db25e2a328dd748cb03ae5ba57c0f Mon Sep 17 00:00:00 2001 From: ewowi Date: Fri, 13 Nov 2026 13:30:37 +0100 Subject: [PATCH 4/5] Harden the desktop update after the pre-merge review A desktop's update now refuses a malformed digest before it downloads, stops at 256 MB, works on a file system without hard links, and leaves a program run as root to be updated by hand. The Windows CI check prints its log, and the docs say what a desktop install trusts and what stops the Linux service. KPI: 256lights | Desktop:2545KB | tick:12/1/123/124282/33/26/17/436/11/11/12us(FPS:83333/1000000/8130/8/30303/38461/58823/2293/90909/90909/83333) | ESP32:2316KB | src:393(99554) | test:292(70180) | functions-over:333 Commit: src +71/-19 lines | test +45/-10 lines | duplicated lines 3186 -> 3186 (+0) The per-scenario worst ticks are this Mac paging with its swap nearly full: Modifiers_reshape_the_mapping read 124282 us in this run and 45 us on an immediate rerun, while Layers_stack_and_blend_live swung the other way; no tick-path code changed. **Core** - HttpServerModule: readDigest refuses a sha256 that is not 64 hex digits with 400 before the install starts; the digest goes to the platform as read, empty meaning none. hex.h gains isHexDigits, which the desktop's digest check uses too. **Platform** - The update mode is computed once for the running program, as the Firmware card shows it, so Windows no longer writes a probe file on every /api/system; a program run as root is updated by hand. - swapIn keeps a copy as .prev where the file system has no hard links (FAT, exFAT, SMB). - An image is capped at 256 MB, at otaBegin and as it is written; the progress line counts against the download's size; sayProgress extracted. - The encoder's wake pipe closes on exec. - ProgramImage.h moves from src/core/util to src/platform/desktop, its one user's platform. **Scripts/MoonDeck** - release_stage.py writes its output as UTF-8, so the Windows smoke-update prints the program's log; the smoke copy is named MoonLight-smoke, so its cleanup stops no other MoonLight. **Tests** - A firmware URL with a malformed sha256 is refused with 400; an image larger than a program could be is refused before a byte is written. The self-update tests skip as root as well as in a container. - Desktop 3271/3271, JS 397, Python 765; the Firmware, Network identity and System identity scenarios pass. **Docs/CI** - security.md: installing firmware on a desktop runs a program as the user running MoonLight. installing-on-linux: three failed starts in two minutes stop the service until it is started by hand. MIGRATING: a drop-in that sets ExecStart goes with the hand-written unit. Three prose negations reworded (the code scanning notes on the PR). The plan links ProgramImage.h at its new path. **Reviews** - Reviewer, a LAN host can install any program on a self desktop: documented in security.md; signing stays in the plan's backlog. - Reviewer, Windows writes a probe file per /api/system: done, the mode is computed once. - Reviewer, hard links fail on FAT and SMB: done, a copy instead. - Reviewer, the Windows smoke result unknown: skipped, the run printed "the program installed itself and restarted on it" before the log print failed; the macOS .app bundle hypothesis stays open in the plan. - Reviewer, a confirmed program's crash loop stops the service silently: done, documented with the command. - Reviewer, root reports self: done, root is updated by hand. - Reviewer, a malformed sha256 refused only after the download: done, 400 up front, tested. - Reviewer, no size bound on an image: done, 256 MB, tested. - Reviewer, ProgramImage.h is platform code in core: done, moved. - Reviewer, the two Dockerfile recipes diverge: skipped, the release recipe stages the package it just built, whose layout is fixed; the public Dockerfile builds any tag. - Reviewer, snprintf of a literal into the update control: skipped, it needs a const char* read-only control, a change to the control system of its own. - Reviewer, no total in the URL install's progress: done. - Reviewer, the encoder wake pipe survives exec: done. - CodeRabbit, widen the start limit: skipped, OnFailure fires on the first failed start (verified on .204, rollback within a second); a wider limit would only stop a confirmed program's crash loop sooner. - CodeRabbit, taskkill by image name: done, the smoke copy has a name of its own. - Code scanning (check_prose), four negations: three reworded; MIGRATING's *nothing* is the Action legend's defined term. - check_taglines, the classic and MoonBase freshness, --no-jit, Improv, GCC and the performance snapshot: SKIP, README, CLAUDE.md, moonbase/, MoonLive and Improv are untouched, CI's GCC build passes, and the tick path is unchanged. Co-Authored-By: Claude Opus 5.5 --- docs/explanation/security.md | 2 +- docs/reference/MIGRATING.md | 3 +- docs/reference/installing-on-linux.md | 2 +- docs/reference/metrics/prose.json | 4 +- docs/reference/metrics/repo-health.json | 32 ++++++------ docs/reference/metrics/repo-health.md | 46 ++++++++-------- ...9 - Devices, groups and the network map.md | 2 +- moondeck/ci/release_stage.py | 5 +- src/core/system/HttpServerModule.cpp | 16 ++++-- src/core/system/HttpServerModule.h | 4 ++ src/core/util/hex.h | 8 +++ .../util => platform/desktop}/ProgramImage.h | 0 src/platform/desktop/platform_desktop.cpp | 6 ++- .../desktop/platform_desktop_update.cpp | 52 +++++++++++++++---- src/platform/platform.h | 2 +- src/platform/teensy/platform_teensy_ota.cpp | 2 +- test/js/installer-desktop-download.test.mjs | 2 +- .../core/unit_HttpServerModule_routes.cpp | 19 +++++++ test/unit/core/unit_ProgramImage.cpp | 4 +- test/unit/core/unit_platform_update.cpp | 30 ++++++++--- 20 files changed, 165 insertions(+), 76 deletions(-) rename src/{core/util => platform/desktop}/ProgramImage.h (100%) diff --git a/docs/explanation/security.md b/docs/explanation/security.md index 2ef380f90..93f8569bf 100644 --- a/docs/explanation/security.md +++ b/docs/explanation/security.md @@ -8,7 +8,7 @@ MoonLight trusts its local network. Every client that reaches a device can read ## The trust boundary is the network -A device serves one plain HTTP port: 80 on a board, 8080 on a desktop. The REST API, the `/ws` state socket and the `/wsp` preview socket all live there, and every route answers every client on the network. A client can set any control, add or delete modules, read and write files, install firmware, reboot the device and reset it to factory settings. The routes: [the REST API](../reference/api.md). +A device serves one plain HTTP port: 80 on a board, 8080 on a desktop. The REST API, the `/ws` state socket and the `/wsp` preview socket all live there, and every route answers every client on the network. A client can set any control, add or delete modules, read and write files, install firmware (on a desktop, a program that then runs as the user running MoonLight), reboot the device and reset it to factory settings. The routes: [the REST API](../reference/api.md). A web page from outside the network is refused. A browser names the page a request comes from in its `Origin` header, and a device answers 403, the WebSocket included, when that page's host is not on the network: a private, shared (RFC 6598) or loopback address, `localhost`, a `.local`, `.home.arpa` or `.internal` name, or a name without a dot; a page the device served, called under the name it was opened by, passes as well. So a site from the internet cannot use a browser on the network against a device. diff --git a/docs/reference/MIGRATING.md b/docs/reference/MIGRATING.md index c95e69e82..cf4a5914d 100644 --- a/docs/reference/MIGRATING.md +++ b/docs/reference/MIGRATING.md @@ -39,11 +39,12 @@ A desktop build names its firmware `desktop--`, such as `desktop-linux The package now installs and starts `moonlight.service`, run as its own `moonlight` user from `/var/lib/moonlight/bin`, a folder that user owns, so the Devices map updates it like a board. The program moves from `/usr/bin/MoonLight` to `/usr/lib/moonlight/MoonLight`, the copy the service starts from, and the menu entry opens `http://localhost:8080/`. -A hand-written `/etc/systemd/system/moonlight.service` overrides the packaged one and names the old path, so remove it before upgrading: +A hand-written `/etc/systemd/system/moonlight.service`, or a drop-in in `/etc/systemd/system/moonlight.service.d/` that sets `ExecStart`, overrides the packaged one and names the old path, so remove it before upgrading: ``` sudo systemctl disable --now moonlight sudo rm /etc/systemd/system/moonlight.service +sudo rm -r /etc/systemd/system/moonlight.service.d # where a drop-in sets ExecStart ``` Settings now live in `/var/lib/moonlight/MoonLight/`. Copy the old user's across once, `pi` here, after the upgrade: diff --git a/docs/reference/installing-on-linux.md b/docs/reference/installing-on-linux.md index 484ecf850..7e4f500ff 100644 --- a/docs/reference/installing-on-linux.md +++ b/docs/reference/installing-on-linux.md @@ -27,7 +27,7 @@ The package installs MoonLight as `moonlight.service` and starts it, so the mach **Updating, as a board updates.** The service owns the folder its program runs from, so it installs a new version of itself through the same routes, Firmware card and map as a board, from the release's `firmware-desktop-linux--v….bin`. It checks the image is a program for this processor and, from the map, against the SHA-256 GitHub publishes for it; it keeps the running program as `MoonLight.prev`, puts the new one in its place and restarts on it in place, the same process systemd started. Its `/api/system` says `update: self`. A package-managed program that updates itself is uncommon on Linux, where the textbook is an APT repository; that needs root to update from the map, and the network-facing service stays unprivileged, so the service owns a copy of the program the package seeds, as Syncthing does. -**Rolling back.** A new program stays pending, marked by `MoonLight.pending`, until it has run a minute. A pending program whose start fails gives way to `MoonLight.prev`, through `moonlight-rollback.service`; a confirmed one stays in place through its restarts. An `apt install` is the administrator's choice: it puts the package's program in place, older or newer, and drops `MoonLight.prev`. +**Rolling back.** A new program stays pending, marked by `MoonLight.pending`, until it has run a minute. A pending program whose start fails gives way to `MoonLight.prev`, through `moonlight-rollback.service`; a confirmed one stays in place through its restarts, and three failed starts within two minutes stop the service until `sudo systemctl start moonlight`. An `apt install` is the administrator's choice: it puts the package's program in place, older or newer, and drops `MoonLight.prev`. **Running it by hand**, to see its log in a terminal: `sudo systemctl stop moonlight`, then `/usr/lib/moonlight/MoonLight`, which keeps its settings in your own `~/.local/share/MoonLight/`. That copy sits in a folder only root writes, so it is updated by hand; the tarball unpacked in a folder of your own updates itself, restarting in its terminal window. diff --git a/docs/reference/metrics/prose.json b/docs/reference/metrics/prose.json index 6b319bc55..7efed5f45 100644 --- a/docs/reference/metrics/prose.json +++ b/docs/reference/metrics/prose.json @@ -1,9 +1,9 @@ { - "(total)": 6885, + "(total)": 6882, "MoonLight.EmDash": 989, "MoonLight.LinkText": 7, "MoonLight.NegatedHeading": 5, - "MoonLight.Negation": 5042, + "MoonLight.Negation": 5039, "MoonLight.SectionLength": 31, "MoonLight.SelfReference": 14, "MoonLight.Spelling": 39, diff --git a/docs/reference/metrics/repo-health.json b/docs/reference/metrics/repo-health.json index 7e37de804..4031ec182 100644 --- a/docs/reference/metrics/repo-health.json +++ b/docs/reference/metrics/repo-health.json @@ -1,7 +1,7 @@ { - "commit": "1f4285222", + "commit": "4bc8eab67", "flash": { - "esp32s3-n16r8": 2493552, + "esp32s3-n16r8": 2493696, "esp32": 2283888, "esp32p4rev1-eth": 2617552, "esp32p4rev1-eth-wifi": 2876672, @@ -14,7 +14,7 @@ "esp32p4rev3-eth": 2615456, "esp32s3-zero": 2125456, "esp32-pico": 2432432, - "desktop-macos": 2605920, + "desktop-macos": 2606432, "desktop-windows": 1925120, "esp32c3": 2405632, "desktop-linux": 2548000, @@ -800,8 +800,8 @@ } }, "desktop-macos": { - "tick_us": 8, - "fps": 125000, + "tick_us": 12, + "fps": 83333, "scenario_p50": { "Effects_pipeline_builds_and_renders": { "p50": 7, @@ -841,16 +841,16 @@ } }, "loc": { - "core": 37904, + "core": 37845, "light": 35148, - "platform": 25826, + "platform": 25937, "ui": 14559, - "test": 86818, - "moondeck": 35531 + "test": 86853, + "moondeck": 35606 }, "comments": { "core": { - "lines": 7447, + "lines": 7428, "ratio": 0.218 }, "light": { @@ -858,7 +858,7 @@ "ratio": 0.233 }, "platform": { - "lines": 4522, + "lines": 4547, "ratio": 0.196 }, "ui": { @@ -866,28 +866,28 @@ "ratio": 0.264 }, "test": { - "lines": 7974, + "lines": 7975, "ratio": 0.106 }, "moondeck": { - "lines": 4509, + "lines": 4513, "ratio": 0.146 } }, "tests": { - "cases": 3261, + "cases": 3263, "scenarios": 37 }, "docs": { "md_files": 122, - "md_lines": 19767, + "md_lines": 19769, "plans_files": 14, "backlog_lines": 2835, "lessons_lines": 455, "claude_md_lines": 139 }, "complexity": { - "functions": 6310, + "functions": 6313, "over_threshold": 333, "worst_ccn": 128 } diff --git a/docs/reference/metrics/repo-health.md b/docs/reference/metrics/repo-health.md index 417feb075..fb07f1306 100644 --- a/docs/reference/metrics/repo-health.md +++ b/docs/reference/metrics/repo-health.md @@ -1,6 +1,6 @@ # Repo health -Measured at `1f4285222`. Generated by [`moondeck/check/repo_health.py`](../../../moondeck/check/repo_health.py) on every KPI-gate run. **Do not edit by hand.** +Measured at `4bc8eab67`. Generated by [`moondeck/check/repo_health.py`](../../../moondeck/check/repo_health.py) on every KPI-gate run. **Do not edit by hand.** Current state only; the trend is this file's git history (`git log -p docs/reference/metrics/repo-health.md`). Nothing here fails a build: the numbers make growth visible, the judgment stays human. @@ -8,7 +8,7 @@ Current state only; the trend is this file's git history (`git log -p docs/refer | Metric | v7.0.0 | Now | Change | |---|---:|---:|---:| -| flash desktop-macos | 2,221 KB | 2,545 KB | +323.6 KB (+14.6%) | +| flash desktop-macos | 2,221 KB | 2,545 KB | +324.1 KB (+14.6%) | | flash esp32 | 2,095 KB | 2,230 KB | +135.5 KB (+6.5%) | | flash esp32-16mb | 2,343 KB | 2,210 KB | -133.1 KB (-5.7%) | | flash esp32-eth | 1,642 KB | 1,827 KB | +185.6 KB (+11.3%) | @@ -19,7 +19,7 @@ Current state only; the trend is this file's git history (`git log -p docs/refer | flash esp32p4rev1-eth-wifi | 2,606 KB | 2,809 KB | +202.8 KB (+7.8%) | | flash esp32p4rev3-eth | 2,357 KB | 2,554 KB | +196.7 KB (+8.3%) | | flash esp32p4rev3-eth-wifi | - | 2,612 KB | new | -| flash esp32s3-n16r8 | 2,300 KB | 2,435 KB | +134.9 KB (+5.9%) | +| flash esp32s3-n16r8 | 2,300 KB | 2,435 KB | +135.0 KB (+5.9%) | | flash esp32s3-n32r16 | 2,322 KB | 2,431 KB | +109.3 KB (+4.7%) | | flash esp32s3-n8r2 | - | 2,194 KB | new | | flash esp32s3-n8r8 | 2,277 KB | 2,301 KB | +23.8 KB (+1.0%) | @@ -29,14 +29,14 @@ Current state only; the trend is this file's git history (`git log -p docs/refer | free heap esp32 | 33,034 KB | 33,034 KB | +0.0 KB (+0.0%) | | tick Effects_pipeline_builds_and_renders (desktop-macos) | 7 µs | 7 µs | +0 (+0.0%) | | tick Layouts_resize_reallocates_live (desktop-macos) | 10 µs | 10 µs | +0 (+0.0%) | -| lines core | 33,397 | 37,904 | +4,507 (+13.5%) | +| lines core | 33,397 | 37,845 | +4,448 (+13.3%) | | lines light | 31,681 | 35,148 | +3,467 (+10.9%) | -| lines platform | 22,993 | 25,826 | +2,833 (+12.3%) | +| lines platform | 22,993 | 25,937 | +2,944 (+12.8%) | | lines ui | 13,043 | 14,559 | +1,516 (+11.6%) | -| lines test | 76,140 | 86,818 | +10,678 (+14.0%) | -| lines moondeck | 32,680 | 35,531 | +2,851 (+8.7%) | +| lines test | 76,140 | 86,853 | +10,713 (+14.1%) | +| lines moondeck | 32,680 | 35,606 | +2,926 (+9.0%) | | code findings | 1,642 | 1,607 | -35 (-2.1%) | -| unit tests | 2,836 | 3,261 | +425 (+15.0%) | +| unit tests | 2,836 | 3,263 | +427 (+15.1%) | | scenarios | 36 | 37 | +1 (+2.8%) | Against the release `v7.0.0`, the fixed point a user compares with; the tables below compare against the last commit. A firmware not rebuilt since `v7.0.0` is left out, and a measure the release did not record yet, such as the code check, joins from the next one. @@ -46,7 +46,7 @@ Against the release `v7.0.0`, the fixed point a user compares with; the tables b | Target | Flash | Capacity | Used | Built | |---|---:|---:|---:|:--:| | desktop-linux | 2,488 KB | - | - | **STALE 32d** | -| desktop-macos | 2,545 KB (+96 B) ⚠ | - | - | yes | +| desktop-macos | 2,545 KB (+512 B) ⚠ | - | - | yes | | desktop-windows | 1,880 KB | - | - | **STALE 44d** | | esp32 | 2,230 KB | 2,496 KB | 89% | carried 1d | | esp32-16mb | 2,210 KB | 4,096 KB | 54% | **STALE 14d** | @@ -55,10 +55,10 @@ Against the release `v7.0.0`, the fixed point a user compares with; the tables b | esp32-wrover | 2,145 KB | 2,496 KB | 86% | **STALE 14d** | | esp32c3 | 2,349 KB | 2,496 KB | 94% | carried 1d | | esp32p4rev1-eth | 2,556 KB | 4,096 KB | 62% | carried 1d | -| esp32p4rev1-eth-wifi | 2,809 KB | 4,096 KB | 69% | yes | +| esp32p4rev1-eth-wifi | 2,809 KB | 4,096 KB | 69% | carried 0d | | esp32p4rev3-eth | 2,554 KB | 4,096 KB | 62% | carried 2d | | esp32p4rev3-eth-wifi | 2,612 KB | 4,096 KB | 64% | **STALE 14d** | -| esp32s3-n16r8 | 2,435 KB (+32 B) ⚠ | 4,096 KB | 59% | yes | +| esp32s3-n16r8 | 2,435 KB (+144 B) ⚠ | 4,096 KB | 59% | yes | | esp32s3-n32r16 | 2,431 KB | 4,096 KB | 59% | carried 3d | | esp32s3-n8r2 | 2,194 KB | 3,072 KB | 71% | **STALE 14d** | | esp32s3-n8r8 | 2,301 KB | 3,072 KB | 75% | **STALE 14d** | @@ -73,13 +73,11 @@ Against the release `v7.0.0`, the fixed point a user compares with; the tables b | Target | Net | New | Removed | Grown | Shrunk | Other | Since | |---|---:|---:|---:|---:|---:|---:|---| -| esp32p4rev1-eth-wifi | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | `1f4285222` | -| esp32s3-n16r8 | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | +0.0 KB | `1f4285222` | +| esp32s3-n16r8 | +0.1 KB | +0.1 KB | +0.0 KB | +0.0 KB | -0.0 KB | +0.0 KB | `4bc8eab67` | Largest moves: -- **esp32p4rev1-eth-wifi**: none -- **esp32s3-n16r8**: `mm::ui::file1` +30 B +- **esp32s3-n16r8**: `mm::HttpServerModule::readDigest(mm::platform::TcpConnection` +118 B, `mm::HttpServerModule::handleFirmwareUrl(mm::platform::TcpCon` -12 B What this commit's flash went to, symbol by symbol against the measurement at `Since`. New is new functionality, removed and shrunk are what a cleanup saved, and other is what no symbol owns, such as merged strings and alignment. A rename reads as removed plus new, which the largest moves expose. @@ -88,7 +86,7 @@ What this commit's flash went to, symbol by symbol against the measurement at `S | Target | Tick | FPS | |---|---:|---:| | desktop-linux | 5 µs | 200,000 | -| desktop-macos | 8 µs | 125,000 | +| desktop-macos | 12 µs (+4 µs) ⚠ | 83,333 (−41,667) ⚠ | | desktop-windows | 4 µs | 250,000 | | esp32 | 1,716 µs | 582 | @@ -145,25 +143,25 @@ These build a bare pipeline with no optional modules, so a change here is a chan | Area | Lines | Comments | Comment share | |---|---:|---:|---:| -| core | 37,904 (+5) ⚠ | 7,447 | 21.8 % | +| core | 37,845 (−59) ✓ | 7,428 | 21.8 % | | light | 35,148 | 7,277 | 23.3 % | -| platform | 25,826 (+21) ⚠ | 4,522 | 19.6 % | -| ui | 14,559 (+2) ⚠ | 3,626 | 26.4 % | -| test | 86,818 (+20) ⚠ | 7,974 | 10.6 % | -| moondeck | 35,531 | 4,509 | 14.6 % | +| platform | 25,937 (+111) ⚠ | 4,547 | 19.6 % | +| ui | 14,559 | 3,626 | 26.4 % | +| test | 86,853 (+35) ⚠ | 7,975 | 10.6 % | +| moondeck | 35,606 (+75) ⚠ | 4,513 | 14.6 % | ## Tests | Kind | Count | |---|---:| -| unit cases | 3,261 (+1) ✓ | +| unit cases | 3,263 (+2) ✓ | | scenarios | 37 | ## Complexity | Metric | Value | |---|---:| -| functions | 6,310 (+1) ✓ | +| functions | 6,313 (+3) ✓ | | over threshold | 333 | | worst CCN | 128 | @@ -172,7 +170,7 @@ These build a bare pipeline with no optional modules, so a change here is a chan | Metric | Value | |---|---:| | markdown files | 122 | -| markdown lines | 19,767 (−6) ✓ | +| markdown lines | 19,769 (+2) ⚠ | | plan files | 14 | | backlog lines | 2,835 | | lessons lines | 455 | diff --git a/docs/work/Plan-20261109 - Devices, groups and the network map.md b/docs/work/Plan-20261109 - Devices, groups and the network map.md index e1719a6e8..cdc4fe7f7 100644 --- a/docs/work/Plan-20261109 - Devices, groups and the network map.md +++ b/docs/work/Plan-20261109 - Devices, groups and the network map.md @@ -180,7 +180,7 @@ Asked by the product owner (2026-11-12): one way to update any MoonLight device, - **Rollback is systemd's, for the service, by boot assessment.** The swap leaves `MoonLight.pending` beside the program, and the new program deletes it once it has stayed up `kBootStableMs`, the minute a board's boot record uses: systemd's Automatic Boot Assessment, for one program. `OnFailure=moonlight-rollback.service`, which systemd starts on every failed start: a pending program that fails, a crash before `main` included, is replaced by `.prev` and the service started again; a confirmed one is left alone, so restarts of a good program never undo it. `StartLimitBurst=3` within `StartLimitIntervalSec=120` stops a program that keeps failing after that. Elsewhere `.prev` stays beside the program, to rename back by hand. - **The Firmware card on a desktop** installs as a board's does, by URL from its picker or by upload, when the update mode is `self`; otherwise it offers the download. The shared install picker tells a desktop's own image (`firmware-desktop-…bin`) from the installer's download of the same name. - **A hypothesis to test first:** a macOS bundle whose inner binary was replaced still launches, Terminal running the binary itself rather than the sealed bundle. -- **Built (2026-11-13).** The image writer, the swap and the restart in [platform_desktop_update.cpp](../../src/platform/desktop/platform_desktop_update.cpp), the header vetting in [ProgramImage.h](../../src/core/util/ProgramImage.h), the download through the one HTTP client (`httpDownload`), the firmware name from CMake, the sockets closed on exec, `hasOta` gone, the bare programs staged by `package_desktop.py`, the Firmware card installing on `self` and offering the download otherwise, the picker telling a desktop's program from its download, the live scenario leaving a desktop to restart itself. Tests: `unit_platform_update.cpp`, `unit_ProgramImage.cpp`, `ui-update-asset.test.mjs`, `test_package_deb.py`. CodeRabbit's 15 findings and a CodeQL alert fixed in the PR's second commit, among them the Linux restart, which read `/proc/self/exe` after the swap and found the old program "(deleted)". +- **Built (2026-11-13).** The image writer, the swap and the restart in [platform_desktop_update.cpp](../../src/platform/desktop/platform_desktop_update.cpp), the header vetting in [ProgramImage.h](../../src/platform/desktop/ProgramImage.h), the download through the one HTTP client (`httpDownload`), the firmware name from CMake, the sockets closed on exec, `hasOta` gone, the bare programs staged by `package_desktop.py`, the Firmware card installing on `self` and offering the download otherwise, the picker telling a desktop's program from its download, the live scenario leaving a desktop to restart itself. Tests: `unit_platform_update.cpp`, `unit_ProgramImage.cpp`, `ui-update-asset.test.mjs`, `test_package_deb.py`. CodeRabbit's 15 findings and a CodeQL alert fixed in the PR's second commit, among them the Linux restart, which read `/proc/self/exe` after the swap and found the old program "(deleted)". - **Removes:** the archive path, the reboot button hidden on desktops, `unknown` as a desktop's firmware name, the Firmware card's desktop download in place of an install, MoonWalk's hand-written unit steps. **Fixed on the way:** the reboot button hidden for a container (`^docker$` never matched `docker-arm64`). - **Costs** a MIGRATING entry: the hand-written unit goes, the program moves from `/usr/bin` to `/usr/lib/moonlight`, and settings move from the old user's home to `/var/lib/moonlight` with one copy command. The Dockerfile copies the program from the new seed path. Unblocks step 9 for a group that mixes boards and desktops. - **Tested (2026-11-13), by the product owner and the agent.** This Mac: an upload installs and restarts in place, same PID, `.pending` cleared after a minute; a board image refused as no program; a URL with a wrong digest refused, with the right one installed and restarted. .204, the Raspberry Pi 5, from the PR's CI `.deb`: the service runs as `moonlight` holding `CAP_NET_RAW` alone, `update: self`; an upload restarts it in place, same PID, no start counted; `/bin/false` installed as an update rolls back to `.prev` within a second; an `apt install` puts the package's program back and drops `.prev`; HDMI shows the effect under the service user. .156, the NanoPi: the PR's program in its container image says `update: host`, refuses both install routes with 501 and restarts in place. The old hand-written unit and a source-build drop-in had to go first, which the MIGRATING entry covers for the unit. CI on the PR: every desktop compiles, Linux and Windows included. diff --git a/moondeck/ci/release_stage.py b/moondeck/ci/release_stage.py index 1439fab7f..6cef27e13 100644 --- a/moondeck/ci/release_stage.py +++ b/moondeck/ci/release_stage.py @@ -297,7 +297,8 @@ def smoke_update(program: Path) -> None: port = 18081 base = f"http://127.0.0.1:{port}" work = Path(tempfile.mkdtemp(prefix="mm-smoke-update-")) - installed = work / ("MoonLight.exe" if os.name == "nt" else "MoonLight") + # A name of its own, so the Windows cleanup by name stops this run and no MoonLight the machine runs besides. + installed = work / ("MoonLight-smoke.exe" if os.name == "nt" else "MoonLight-smoke") shutil.copy2(program, installed) log_path = work / "run.log" with log_path.open("wb") as log: @@ -369,6 +370,8 @@ def gcc_version() -> str: def main() -> int: + # UTF-8 whatever the runner's console: a Windows runner's cp1252 cannot print the program's own log, which says "→". + sys.stdout.reconfigure(encoding="utf-8", errors="replace") parser = argparse.ArgumentParser(description=__doc__) sub = parser.add_subparsers(dest="command", required=True) tag = sub.add_parser("tag", help="Emit tag=.") diff --git a/src/core/system/HttpServerModule.cpp b/src/core/system/HttpServerModule.cpp index c06509ac1..a2bad637f 100644 --- a/src/core/system/HttpServerModule.cpp +++ b/src/core/system/HttpServerModule.cpp @@ -18,6 +18,7 @@ #include "core/util/CaptivePortal.h" // the redirect a phone on the access point follows to the UI #include "core/util/fnv.h" // fnv1a: the diff-on-the-wire cache's digest #include "core/system/Sha1.h" +#include "core/util/sha256.h" // kSha256DigestSize: the digest a desktop's install names #include "core/system/Base64.h" #include "core/system/FilesystemModule.h" #include "core/system/FirmwareUpdateModule.h" @@ -2092,6 +2093,14 @@ bool HttpServerModule::readImageUrl(platform::TcpConnection& conn, const char* b return false; } +// One character past the 64 digits, so a longer value shows rather than reading as cut to fit. +bool HttpServerModule::readDigest(platform::TcpConnection& conn, const char* body, char (&sha256)[kDigestText]) { + mm::json::parseString(body, "sha256", sha256, sizeof(sha256)); + if (!sha256[0] || isHexDigits(sha256, kSha256DigestSize * 2)) return true; + sendError(conn, 400, "sha256 must be 64 hex digits"); + return false; +} + // A MoonBase device runs from its one app slot, so the URL is staged in NVS and the device reboots into MoonBase to install; 202 with {"moonbase":true} marks this flow. void HttpServerModule::installThroughMoonBase(platform::TcpConnection& conn, const char* url) { if (!platform::moonbaseStageInstallUrl(url)) { @@ -2123,6 +2132,8 @@ void HttpServerModule::handleFirmwareUrl(platform::TcpConnection& conn, const ch char url[kMaxImageUrl + 2] = {}; if (!readImageUrl(conn, body, url, false)) return; + char sha256[kDigestText] = {}; + if (!readDigest(conn, body, sha256)) return; // A device without TLS has no firmware fetcher at all, so it says so before starting: the UI then fetches the image itself and uploads it. if (!platform::httpsAvailable()) { sendResponse(conn, 501, "application/json", @@ -2141,11 +2152,8 @@ void HttpServerModule::handleFirmwareUrl(platform::TcpConnection& conn, const ch g_otaBytesTotal = 0; FilesystemModule::flushPending(); // the install ends in a restart, which drops settings still waiting for their debounce - // The image's digest, which a desktop checks before it installs its new program; a board's image carries its own. - char sha256[65] = {}; - mm::json::parseString(body, "sha256", sha256, sizeof(sha256)); if (!platform::http_fetch_to_ota(url, g_otaStatus, sizeof(g_otaStatus), - &g_otaBytesRead, &g_otaBytesTotal, nullptr, sha256[0] ? sha256 : nullptr)) { + &g_otaBytesRead, &g_otaBytesTotal, nullptr, sha256)) { sendError(conn, 500, g_otaStatus[0] ? g_otaStatus : "ota start failed"); // the platform's own reason when it wrote one return; } diff --git a/src/core/system/HttpServerModule.h b/src/core/system/HttpServerModule.h index fa7c30c48..95186cdf5 100644 --- a/src/core/system/HttpServerModule.h +++ b/src/core/system/HttpServerModule.h @@ -927,6 +927,10 @@ class HttpServerModule : public MoonModule, public BinaryBroadcaster { static constexpr size_t kMaxImageUrl = 255; /// Read the image URL from `body`, the release's matching radio-chip image when `coprocessorDefault` and none is named; false after answering 400. bool readImageUrl(platform::TcpConnection& conn, const char* body, char (&url)[kMaxImageUrl + 2], bool coprocessorDefault); + /// A SHA-256 written out with one character to spare and the terminator. + static constexpr size_t kDigestText = 66; + /// Read the image's digest from `body`, which a desktop checks before it installs its new program, empty when none is named; false after answering 400. + bool readDigest(platform::TcpConnection& conn, const char* body, char (&sha256)[kDigestText]); /// Stage `url` for MoonBase and reboot into it, which installs it and boots the app again. void installThroughMoonBase(platform::TcpConnection& conn, const char* url); /// Install an uploaded image, the application's (`POST /api/firmware/upload`) or MoonBase's (`POST /api/firmware/moonbase-update`), through a transfer slot. diff --git a/src/core/util/hex.h b/src/core/util/hex.h index a22bd5c49..25e57392e 100644 --- a/src/core/util/hex.h +++ b/src/core/util/hex.h @@ -12,4 +12,12 @@ inline int hexDigit(char h) { return -1; } +/// Whether `text` is exactly `count` hex digits, as a digest written out is. +inline bool isHexDigits(const char* text, unsigned count) { + unsigned n = 0; + for (; text[n]; n++) + if (n == count || hexDigit(text[n]) < 0) return false; + return n == count; +} + } // namespace mm diff --git a/src/core/util/ProgramImage.h b/src/platform/desktop/ProgramImage.h similarity index 100% rename from src/core/util/ProgramImage.h rename to src/platform/desktop/ProgramImage.h diff --git a/src/platform/desktop/platform_desktop.cpp b/src/platform/desktop/platform_desktop.cpp index 6f6d016af..c6cbfcc9a 100644 --- a/src/platform/desktop/platform_desktop.cpp +++ b/src/platform/desktop/platform_desktop.cpp @@ -651,7 +651,7 @@ bool httpsAvailable() MM_NONBLOCKING { // Windows needs no libcurl: WinHTTP ships with the OS, so the send path is always compiled in. return true; #else - return false; // built without libcurl: httpFetch can never succeed + return false; // a build lacking libcurl: httpFetch fails every time #endif } @@ -861,7 +861,7 @@ int httpDownload(const char* url, uint32_t timeoutMs, DownloadChunkFn take, void return transportFetch("GET", url, nullptr, timeoutMs, sink, true); #else (void)timeoutMs; (void)user; (void)total; - return 0; // built without libcurl: a download fails, visibly + return 0; // a build lacking libcurl: the download fails, visibly #endif } @@ -3634,6 +3634,8 @@ static bool spawnEncoderProcess(const char* const argv[], bool captureStdout = f esWake_[0] = esWake_[1] = -1; return true; // the encoder runs; only the frames this thread would publish are absent } + close_on_exec(esWake_[0]); + close_on_exec(esWake_[1]); #endif { std::lock_guard lk(esMutex_); diff --git a/src/platform/desktop/platform_desktop_update.cpp b/src/platform/desktop/platform_desktop_update.cpp index 50e56a886..e723cade7 100644 --- a/src/platform/desktop/platform_desktop_update.cpp +++ b/src/platform/desktop/platform_desktop_update.cpp @@ -7,7 +7,8 @@ #include "host_update.h" #include "platform/platform.h" -#include "core/util/ProgramImage.h" // the header a new program is vetted by, as a board vets its image's +#include "ProgramImage.h" // the header a new program is vetted by, as a board vets its image's +#include "core/util/hex.h" // isHexDigits: a digest's form #include "core/util/sha256.h" // the digest a fetch is checked against #include @@ -25,7 +26,7 @@ #include #else #include // pthread_sigmask: the new program starts with every signal open -#include // access, execv +#include // access, execv, geteuid #endif #ifdef __APPLE__ #include // _NSGetArgc, _NSGetArgv: the command line the restart repeats @@ -40,6 +41,9 @@ namespace { fs::path g_testProgram; // a test's stand-in for the running program, empty for the real one +/// The largest image a desktop takes, some 80 times its program: a URL serving more fills the disk before its header could refuse it. +constexpr uint32_t kMaxProgramBytes = 256u * 1024 * 1024; + /// The system and processor this build runs on, which a new program must be built for. constexpr ProgramSystem kThisSystem = #if defined(_WIN32) @@ -86,7 +90,7 @@ std::vector readHead(const fs::path& path) { /// Whether the file's SHA-256 is the one the release names, compared case-insensitively. bool digestMatches(const fs::path& path, const char* expectedHex) { - if (std::strlen(expectedHex) != kSha256DigestSize * 2) return false; + if (!isHexDigits(expectedHex, kSha256DigestSize * 2)) return false; std::error_code ec; const auto fileBytes = fs::file_size(path, ec); if (ec) return false; @@ -136,6 +140,10 @@ const char* swapIn(const fs::path& next, const fs::path& program) { fs::permissions(next, fs::perms::owner_all | fs::perms::group_read | fs::perms::group_exec | fs::perms::others_read | fs::perms::others_exec, ec); // The running program keeps a second name, so the rollback has it to put back and the program's own name holds a whole program at every moment. fs::create_hard_link(program, prev, ec); + if (ec) { // a file system without hard links (FAT, exFAT, SMB) keeps a copy instead + ec.clear(); + fs::copy_file(program, prev, ec); + } if (ec) return "error: cannot keep the running program as .prev"; fs::rename(next, program, ec); // rename(2), atomic: the name moves from the old program to the new one in one step #endif @@ -151,10 +159,19 @@ struct ProgramWrite { size_t statusLen = 0; uint32_t* bytesOut = nullptr; uint32_t expected = 0; ///< the image's size once known, for the progress line + const uint32_t* total = nullptr; ///< a download's size as its headers name it, where `expected` is unknown up front uint32_t written = 0; std::string sha256; ///< the digest a fetch names, empty for an upload }; +/// The progress line, "of N" once the image's size is known, from the upload's length or the download's headers. +void sayProgress(const ProgramWrite& img) { + if (!img.status || !img.statusLen) return; + const uint32_t expected = img.expected ? img.expected : (img.total ? *img.total : 0); + if (expected) std::snprintf(img.status, img.statusLen, "flashing: %u of %u bytes", static_cast(img.written), static_cast(expected)); + else std::snprintf(img.status, img.statusLen, "flashing: %u bytes", static_cast(img.written)); +} + /// Drop an image that will not be installed. void abandon(OtaWrite& w) { std::unique_ptr img(static_cast(w.impl)); @@ -194,6 +211,7 @@ void fetchTask(const std::string& url, const std::string& fallbackUrl, const std OtaWrite w; if (!otaBegin(w, OtaTarget::App, 0, status, statusLen, bytesRead)) return; static_cast(w.impl)->sha256 = sha256Hex; + static_cast(w.impl)->total = bytesTotal; const int httpStatus = httpDownload(from->c_str(), 10 * 60 * 1000, [](void* user, const char* data, size_t n) { return otaWrite(*static_cast(user), reinterpret_cast(data), n); @@ -241,8 +259,16 @@ void setTestProgramPath(const fs::path& path) { g_testProgram = path; } UpdateMode updateMode() { if (inContainer()) return UpdateMode::Host; - const fs::path program = programPath(); - return !program.empty() && folderWritable(program.parent_path()) ? UpdateMode::Self : UpdateMode::Manual; + const auto modeOf = [](const fs::path& program) { +#ifndef _WIN32 + // Root writes every folder, the package's seed among them, so a program run as root is updated by hand, never on a request from the network. + if (::geteuid() == 0) return UpdateMode::Manual; +#endif + return !program.empty() && folderWritable(program.parent_path()) ? UpdateMode::Self : UpdateMode::Manual; + }; + if (!g_testProgram.empty()) return modeOf(g_testProgram); + static const UpdateMode installed = modeOf(programPath()); // once, as the Firmware card shows it: Windows asks its folder with a probe file + return installed; } bool putProgramInPlace(const fs::path& next, const fs::path& program, const char* sha256Hex, char* status, size_t statusLen) { @@ -292,6 +318,10 @@ bool otaBegin(OtaWrite& w, OtaTarget target, size_t contentLen, char* statusBuf, say(statusBuf, statusBufLen, "error: this program is updated by its host or by hand"); return false; } + if (contentLen > kMaxProgramBytes) { + say(statusBuf, statusBufLen, "error: the image is too large for a program"); + return false; + } auto img = std::unique_ptr(new (std::nothrow) ProgramWrite); if (!img) { say(statusBuf, statusBufLen, "error: out of memory"); return false; } img->program = programPath(); @@ -310,17 +340,17 @@ bool otaBegin(OtaWrite& w, OtaTarget target, size_t contentLen, char* statusBuf, bool otaWrite(OtaWrite& w, const uint8_t* data, size_t len) { auto* img = static_cast(w.impl); if (!img) return false; - if (std::fwrite(data, 1, len, img->file) != len) { - say(img->status, img->statusLen, "error: cannot write the new program"); + const char* failure = len > kMaxProgramBytes - img->written ? "error: the image is too large for a program" + : std::fwrite(data, 1, len, img->file) != len ? "error: cannot write the new program" + : nullptr; + if (failure) { + say(img->status, img->statusLen, failure); abandon(w); return false; } img->written += static_cast(len); if (img->bytesOut) *img->bytesOut = img->written; - if (img->status && img->statusLen) { - if (img->expected) std::snprintf(img->status, img->statusLen, "flashing: %u of %u bytes", static_cast(img->written), static_cast(img->expected)); - else std::snprintf(img->status, img->statusLen, "flashing: %u bytes", static_cast(img->written)); - } + sayProgress(*img); return true; } diff --git a/src/platform/platform.h b/src/platform/platform.h index d5f1a240f..8e97f93f7 100644 --- a/src/platform/platform.h +++ b/src/platform/platform.h @@ -913,7 +913,7 @@ constexpr const char* updateModeName(UpdateMode m) { /// How this device takes a new version: a board installs its own images, a desktop installs itself when its program sits in a folder it may write, a container is its host's. UpdateMode updateMode(); -/// Fetch a firmware image from `url`, or `fallbackUrl` where that cannot be opened, and install it, returning at once; a desktop checks `sha256Hex`, a board's image its own. +/// Fetch a firmware image from `url`, or `fallbackUrl` where that cannot be opened, and install it, returning at once; a desktop checks `sha256Hex` when it is non-empty, a board's image its own. bool http_fetch_to_ota(const char* url, char* statusBuf, size_t statusBufLen, uint32_t* bytesReadOut, uint32_t* bytesTotalOut, diff --git a/src/platform/teensy/platform_teensy_ota.cpp b/src/platform/teensy/platform_teensy_ota.cpp index 1554482b5..2784e27c7 100644 --- a/src/platform/teensy/platform_teensy_ota.cpp +++ b/src/platform/teensy/platform_teensy_ota.cpp @@ -4,7 +4,7 @@ /// /// ## One slot, so the copy waits for the reboot /// -/// An ESP32 writes its update into a second app slot; the Teensy's program flash holds one image, so the upload goes into the erased space above it, where nothing runs. +/// An ESP32 writes its update into a second app slot; the Teensy's program flash holds one image, so the upload goes into the erased space above it, idle until the reboot. /// /// The copy over the program happens in `reboot()`, after the HTTP answer is out, in 64 KB blocks, the flash's largest erase, skipping every block the new image shares with the running one. /// diff --git a/test/js/installer-desktop-download.test.mjs b/test/js/installer-desktop-download.test.mjs index 4c2e93864..da3e96c70 100644 --- a/test/js/installer-desktop-download.test.mjs +++ b/test/js/installer-desktop-download.test.mjs @@ -122,7 +122,7 @@ test("an ESP32 device is never offered a desktop archive to flash", () => { } }); -test("a desktop is never offered an ESP32 firmware, downloading or updating itself", () => { +test("a desktop is offered desktop programs alone, downloading or updating itself", () => { // Whichever OS the page runs on, no ESP32 image is a desktop's download or its update. for (const key of ["esp32", "esp32s3-n16r8", "esp32p4", "esp32-eth"]) { assert.equal(isCompatible(DESKTOP_DOWNLOAD, key), false, `download mode must reject ${key}`); diff --git a/test/unit/core/unit_HttpServerModule_routes.cpp b/test/unit/core/unit_HttpServerModule_routes.cpp index 0e84c843b..cc2e68a06 100644 --- a/test/unit/core/unit_HttpServerModule_routes.cpp +++ b/test/unit/core/unit_HttpServerModule_routes.cpp @@ -156,6 +156,25 @@ TEST_CASE("an image URL longer than the MoonBase handoff takes is refused with 4 http.release(); } +// A digest in another form than GitHub's 64 hex digits is refused before the download, rather than failing the install after it. +TEST_CASE("a firmware URL whose sha256 is no 64 hex digits is refused with 400" + * doctest::skip(mm::platform::updateMode() != mm::platform::UpdateMode::Self)) { + mm::HttpServerModule http; + http.port = kTestPort; + http.setup(); + for (const char* digest : {"sha256:0123", "zz", "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef0"}) { + mm::platform::TcpConnection client; + REQUIRE(connectAndAccept(http, client, kTestPort)); + const std::string req = withBody("POST /api/firmware/url", std::string("{\"url\":\"https://example.com/p.bin\",\"sha256\":\"") + digest + "\"}"); + REQUIRE(client.write(reinterpret_cast(req.data()), req.size())); + const std::string reply = readUntilClosed(http, client); + client.close(); + CHECK_MESSAGE(reply.rfind("HTTP/1.1 400", 0) == 0, digest); + CHECK_MESSAGE(reply.find("sha256 must be 64 hex digits") != std::string::npos, digest); + } + http.release(); +} + // A name past what a module holds is refused on every route that names one, rather than cut short into a name that could match another module. TEST_CASE("a module route whose name does not fit is refused with 400 on move, replace, delete and read alike") { RackServer server; diff --git a/test/unit/core/unit_ProgramImage.cpp b/test/unit/core/unit_ProgramImage.cpp index d1f981c65..9663c8976 100644 --- a/test/unit/core/unit_ProgramImage.cpp +++ b/test/unit/core/unit_ProgramImage.cpp @@ -1,11 +1,11 @@ -/// @module core +/// @module platform /// Pins what a desktop program's first bytes say about it, the check a desktop runs before a new version replaces it, as a board checks its image's chip. /// /// An ELF, Mach-O or PE program for the right system and processor passes; another one, a library or an object file is refused with its reason, and a web page, an archive or a cut header is no program. #include "doctest.h" -#include "core/util/ProgramImage.h" +#include "ProgramImage.h" // src/platform/desktop: the one platform that runs programs #include #include diff --git a/test/unit/core/unit_platform_update.cpp b/test/unit/core/unit_platform_update.cpp index 43b9df6e9..14486d4cc 100644 --- a/test/unit/core/unit_platform_update.cpp +++ b/test/unit/core/unit_platform_update.cpp @@ -97,8 +97,17 @@ struct Scratch { } }; -/// A container's program is its host's to replace, so the self-update cases run on a native host only. -const auto kSelfUpdating = doctest::skip(platform::inContainer()); +/// Whether this test runs as root, whose program is updated by hand. +bool runningAsRoot() { +#ifdef _WIN32 + return false; +#else + return ::geteuid() == 0; +#endif +} + +/// A container's program is its host's to replace and root's is updated by hand, so the self-update cases run as a native user only. +const auto kSelfUpdating = doctest::skip(platform::inContainer() || runningAsRoot()); } // namespace @@ -106,11 +115,8 @@ TEST_CASE("a native program in a folder it can write updates itself, one in a fo Scratch s("mode"); CHECK(platform::updateMode() == platform::UpdateMode::Self); #ifndef _WIN32 - // Root may write any folder whatever its mode, so this half holds for an ordinary user only. - if (::geteuid() != 0) { - fs::permissions(s.dir, fs::perms::owner_read | fs::perms::owner_exec); - CHECK(platform::updateMode() == platform::UpdateMode::Manual); - } + fs::permissions(s.dir, fs::perms::owner_read | fs::perms::owner_exec); + CHECK(platform::updateMode() == platform::UpdateMode::Manual); #endif CHECK(std::string(platform::updateModeName(platform::UpdateMode::Self)) == "self"); CHECK(std::string(platform::updateModeName(platform::UpdateMode::Host)) == "host"); @@ -166,3 +172,13 @@ TEST_CASE("a desktop has one program: MoonBase and the radio chip are refused") CHECK_FALSE(platform::otaBegin(w, platform::OtaTarget::Coprocessor, 0, status, sizeof(status), &written)); CHECK(std::string(status).rfind("error:", 0) == 0); } + +TEST_CASE("an image larger than a program could be is refused before a byte is written" * kSelfUpdating) { + Scratch s("too_large"); + char status[64] = {}; + uint32_t written = 0; + platform::OtaWrite w; + CHECK_FALSE(platform::otaBegin(w, platform::OtaTarget::App, size_t{1} << 30, status, sizeof(status), &written)); + CHECK(std::string(status).find("too large") != std::string::npos); + CHECK_FALSE(fs::exists(s.dir / "MoonLight.next")); +} From 277261ffb3b0558b8923c4214f254c5dc395f711 Mon Sep 17 00:00:00 2001 From: ewowi Date: Fri, 13 Nov 2026 13:46:41 +0100 Subject: [PATCH 5/5] Reset the start limit before restarting a stopped Linux service The Linux page's restart command for a service its start limit stopped now runs `systemctl reset-failed` first: within the limit's two minutes, systemd refuses a manual start as well. KPI: not run, a documentation line only. **Docs/CI** - installing-on-linux: `sudo systemctl reset-failed moonlight && sudo systemctl start moonlight`, as the rollback script itself does before its start. **Reviews** - CodeRabbit, the manual start is subject to the start limit: done. - Every other check: SKIP, a command added to one code span. Co-Authored-By: Claude Opus 5.5 --- docs/reference/installing-on-linux.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/reference/installing-on-linux.md b/docs/reference/installing-on-linux.md index 7e4f500ff..daf44792f 100644 --- a/docs/reference/installing-on-linux.md +++ b/docs/reference/installing-on-linux.md @@ -27,7 +27,7 @@ The package installs MoonLight as `moonlight.service` and starts it, so the mach **Updating, as a board updates.** The service owns the folder its program runs from, so it installs a new version of itself through the same routes, Firmware card and map as a board, from the release's `firmware-desktop-linux--v….bin`. It checks the image is a program for this processor and, from the map, against the SHA-256 GitHub publishes for it; it keeps the running program as `MoonLight.prev`, puts the new one in its place and restarts on it in place, the same process systemd started. Its `/api/system` says `update: self`. A package-managed program that updates itself is uncommon on Linux, where the textbook is an APT repository; that needs root to update from the map, and the network-facing service stays unprivileged, so the service owns a copy of the program the package seeds, as Syncthing does. -**Rolling back.** A new program stays pending, marked by `MoonLight.pending`, until it has run a minute. A pending program whose start fails gives way to `MoonLight.prev`, through `moonlight-rollback.service`; a confirmed one stays in place through its restarts, and three failed starts within two minutes stop the service until `sudo systemctl start moonlight`. An `apt install` is the administrator's choice: it puts the package's program in place, older or newer, and drops `MoonLight.prev`. +**Rolling back.** A new program stays pending, marked by `MoonLight.pending`, until it has run a minute. A pending program whose start fails gives way to `MoonLight.prev`, through `moonlight-rollback.service`; a confirmed one stays in place through its restarts, and three failed starts within two minutes stop the service until `sudo systemctl reset-failed moonlight && sudo systemctl start moonlight`. An `apt install` is the administrator's choice: it puts the package's program in place, older or newer, and drops `MoonLight.prev`. **Running it by hand**, to see its log in a terminal: `sudo systemctl stop moonlight`, then `/usr/lib/moonlight/MoonLight`, which keeps its settings in your own `~/.local/share/MoonLight/`. That copy sits in a folder only root writes, so it is updated by hand; the tarball unpacked in a folder of your own updates itself, restarting in its terminal window.