diff --git a/AGENTS.md b/AGENTS.md index 493f999..55e6b38 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -473,10 +473,18 @@ Real fix: `sim_runtime_dir/0` should detect `ios/build.zig` and use `run-as` ERTS probe, exists because after an APK reinstall the root-pushed tree can carry the previous install's categories, which `run-as` can't read (MOB-478). -- **Physical iOS BEAM overrides must be exact and self-verifying.** The app - prefers `Documents/otp/` over its complete signed bundle. Replace that - directory rather than incrementally merging it, require `.beam` before - transfer, and verify the received bootstrap bytes before restarting. +- **Physical iOS BEAM uploads never remove anything and never land in place.** + `devicectl --remove-existing-content` empties the app's whole data + container (Documents, Library, tmp), whatever `--destination` says + (MOB-408); never pass it. For mob ≥ 0.9.20 upload to a fresh + `Documents/otp/.versions/`, verify the received bootstrap bytes, + then copy `/.mob_complete`; the app loads only committed versions and + deletes the old ones itself (devicectl has no delete verb). Older runtimes + get a merge into `Documents/otp/`. Require `.beam` before + transfer and verify it before restarting, either way, and after the + relaunch confirm `Documents/mob_diag_beams_dir.txt` names the directory + written: which one the app loads is the runtime's choice, not mob_dev's. See + `decisions/2026-10-15-ios-deploy-never-empties-the-container.md`. - **iOS bundle ids resolve through one function per side, never `bundle_id/0`.** Consumers (deploy, connect, provision, battery bench) use `MobDev.Config.ios_bundle_id/0`; the build (sim bundle, device bundle, diff --git a/CHANGELOG.md b/CHANGELOG.md index 8cab0fa..3ab8dd1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,35 @@ +## [Unreleased] + +### Fixed + +- **A BEAM deploy to a physical iPhone no longer erases the app's data + (MOB-408).** The deploy copied the BEAMs with `xcrun devicectl device copy + to ... --remove-existing-content true`, meant to replace + `Documents/otp/`. devicectl instead empties the app's whole data + container apart from the destination: every deploy deleted the app's SQLite + database, sessions, pairing state, the rest of `Documents/`, and `Library/` + and `tmp/`. Copies now only add and overwrite. For an app on mob ≥ 0.9.20 + each deploy goes to a new `Documents/otp/.versions//`, committed by + a `.mob_complete` marker once the bootstrap BEAM reads back correctly; the + app loads the newest committed version and deletes the older ones, so an + interrupted deploy leaves the previous code running. For an older runtime + the BEAMs are merged into `Documents/otp/`; there a module removed + from the project stays loadable until the app is reinstalled or moves to + mob ≥ 0.9.20, and an interrupted copy can still leave old and new BEAMs + mixed (the error says so). The new layout is used when the project's mob + ships `ios/mob_code_versions.h` and the phone has + `Documents/otp/.versions/`, which that runtime creates at each boot. + After relaunching the app, the deploy now waits (up to 20 s) for its + `Documents/mob_diag_beams_dir.txt` to name the directory just written, and + fails, naming the one it booted from, if it doesn't: that is an app whose + native runtime and `:mob` dependency disagree. See + `decisions/2026-10-15-ios-deploy-never-empties-the-container.md`. + **Upgrading:** updating mob_dev stops the wipe, with any mob. For crash-safe + deploys also update mob to 0.9.20 and run `mix mob.deploy --native` once; + that deploy still merges into `Documents/otp/`, and the ones after it + use the numbered directories. Moving an app's mob across 0.9.20 later, in + either direction, needs `--native` too, or fast deploys fail the boot check. + ## [0.7.27] - 2026-10-15 ### Fixed diff --git a/decisions/2026-09-05-dist-deploy-writes-disk-too.md b/decisions/2026-09-05-dist-deploy-writes-disk-too.md index 286b1b3..1caf083 100644 --- a/decisions/2026-09-05-dist-deploy-writes-disk-too.md +++ b/decisions/2026-09-05-dist-deploy-writes-disk-too.md @@ -48,7 +48,13 @@ succeeded. And even over USB the write is an `xcrun devicectl ... --remove-existing-content` replace with no undo: a cable knock mid-copy leaves the app unbootable, with no way back but another successful deploy. A hot load could never damage a device. Making it able to, -silently, on the most-used command in the repo, is not a fix. +silently, on the most-used command in the repo, is not a fix. (Amended by +MOB-408: that replace was worse than stated, since the flag empties the app's +whole data container. From mob_dev 0.7.28 the write never removes anything: +an app on mob ≥ 0.9.20 gets a new committed directory that leaves the +previous code running if interrupted, an older one a merge into +`Documents/otp/`, which can still leave it unbootable. The first reason +stands either way; see `2026-10-15-ios-deploy-never-empties-the-container.md`.) The documented physical-iOS dist workflow is USB *unplugged* anyway, which is precisely the state in which the write cannot run. diff --git a/decisions/2026-09-30-private-dist-cookie-for-every-platform.md b/decisions/2026-09-30-private-dist-cookie-for-every-platform.md index aacea58..353619a 100644 --- a/decisions/2026-09-30-private-dist-cookie-for-every-platform.md +++ b/decisions/2026-09-30-private-dist-cookie-for-every-platform.md @@ -59,7 +59,9 @@ the bundle id. iOS, Android and the Mac-side node all use it. file, so it stays attachable. (Amended by MOB-348: iOS filesystem deploys now also write `mob_dist_cookie` into the beams dir, the simulator runtime dir or the physical iPhone's `Documents/otp//`, and mob ≥ the matching - release reads it, so iOS relaunches stay attachable too.) + release reads it, so iOS relaunches stay attachable too. Amended again by + MOB-408: on a physical iPhone with mob ≥ 0.9.20 that is the deploy's new + `Documents/otp/.versions//`.) - While a legacy app is connected, its node name carries `mob_secret` as a per-node cookie on the Mac. That is the old exposure, limited to that name, and the warning tells the developer how to end it. diff --git a/decisions/2026-10-15-ios-deploy-never-empties-the-container.md b/decisions/2026-10-15-ios-deploy-never-empties-the-container.md new file mode 100644 index 0000000..7878518 --- /dev/null +++ b/decisions/2026-10-15-ios-deploy-never-empties-the-container.md @@ -0,0 +1,90 @@ +# A physical-iPhone deploy never removes anything: numbered uploads, or a merge + +- Date: 2026-10-15 +- Status: accepted +- Linear: MOB-408 + +## Context + +`mix mob.deploy` copied a physical iPhone's BEAMs with +`xcrun devicectl device copy to ... --destination Documents/otp/ +--remove-existing-content true`, replacing the override rather than merging +into it (so removed modules didn't linger), then read back `.beam` to +check it. + +Measured on Kevin's iPhone SE (devicectl 642.16): with that flag, devicectl +empties the app's whole data container except the destination, not the +destination directory its help describes. A merge copy (no flag) to +`Documents/otp/zz` left everything in place; the same copy with the flag left +only `Documents/otp/zz`, deleting `Documents/app.db`, the other Documents files +and directories, `Documents/otp`'s other entries, and the files in `Library/` +and `tmp/`. Reproduced end to end with a blank host on mob 0.9.19 and mob_dev +0.7.26: a sentinel file, a SQLite row, a nested directory, a `Library/` file and +a sibling in `Documents/otp` were all gone after one BEAM-only deploy. Every +deploy erased the app's data, and an interrupted one left the only directory +`mob_beam.m` loads incomplete. + +devicectl has no delete or rename verb, so the Mac can neither clean up nor +swap a directory atomically. `devicectl device info files` lists a directory; +for one that doesn't exist (or an app that isn't installed) it exits 1 with +`CoreDevice.ActionError` code 3 in its JSON. + +## Decision + +Never pass `--remove-existing-content`. Every copy is a merge. Where it goes +depends on the app's runtime: + +- **mob ≥ 0.9.20** (`ios/mob_code_versions.h` in mob; mob's + `decisions/2026-10-15-ios-versioned-code-dirs.md`): list + `Documents/otp/.versions/`, upload to `/` with `N` one above the + highest numeric entry (committed or not, so an upload never lands on an + interrupted one's leftovers), read back `/.beam` and compare, then + copy `/.mob_complete`. The app loads the highest committed `N` and + deletes older ones at boot, so an interrupted deploy leaves the previous + code running. +- **Older runtimes**: merge into `Documents/otp/`, the one directory they + read. A module removed from the project lingers there, and an interrupted + copy can still leave a mix of old and new BEAMs; the error says so. + +The versioned layout is used only when both are true (`ios_code_plan/2`): + +- the project's `:mob` dependency has `ios/mob_code_versions.h`, so the + runtime built from it reads the layout. A file, not a version number, so a + path or git dependency of an unreleased mob counts; +- `Documents/otp/.versions/` exists on the phone, which the runtime + creates at every boot. + +The first alone misses mob updated without `--native` (an old runtime still +on the phone, which would ignore the upload); the second alone misses a phone +that ran a newer mob before a `--native` downgrade. A listing that fails any +other way fails the deploy. + +Both together still guess wrong when the project's mob moves across 0.9.20 +without `--native` on a phone that has `.versions/`: a versioned upload +an old runtime ignores, or a legacy merge that a new runtime's committed +version shadows (and that runtime then deletes). So the deploy checks the +result instead of trusting the guess. Every mob runtime since 0.6.2 writes +the directory it loaded to `Documents/mob_diag_beams_dir.txt` at boot. The +deploy overwrites that file with a placeholder (a legacy deploy expects the +same path the previous boot may have written), relaunches, reads it until it +names the directory just written (up to 20 s), and otherwise fails, naming +where the app booted from and saying to run `mix mob.deploy --native`. No +relaunch (a launch that failed, or `restart: false`) means nothing to check. + +## Consequences + +- No BEAM deploy deletes app data again, whatever mob the app runs. +- A physical-iPhone deploy that relaunched the app and reports success has + had the app boot from the code it wrote. +- The `mix mob.deploy --native` run that first puts mob ≥ 0.9.20 on a phone + still merges into `Documents/otp/` (the new runtime hasn't booted to + create `.versions/`); its restart boots that runtime, later deploys + are versioned, and the first versioned boot deletes the legacy directory. +- Each deploy costs one more devicectl call (the listing, for a project on + mob ≥ 0.9.20), one more for a versioned one (the marker), and the boot + check: about a second over USB in total, more if the app is slow to start. +- Two deploys to one phone at the same moment can both pick the same `N`. + Concurrent deploys could always clobber each other; this doesn't fix that. +- A physical iPhone connected over dist is still only hot-loaded + (`persistable?/1`): a LAN-only device has no devicectl route, and an old + runtime's merge can still break the app mid-copy. diff --git a/lib/mix/tasks/mob.deploy.ex b/lib/mix/tasks/mob.deploy.ex index 551611a..1c38419 100644 --- a/lib/mix/tasks/mob.deploy.ex +++ b/lib/mix/tasks/mob.deploy.ex @@ -139,8 +139,18 @@ defmodule Mix.Tasks.Mob.Deploy do rsync -a / ~/.mob/runtime/ios-sim// xcrun simctl launch - # Any other physical iPhone: replace Documents/otp/, relaunch - xcrun devicectl device copy to --device ... Documents/otp/ + # Any other physical iPhone: upload into a new numbered directory, + # check it, commit it, relaunch, and confirm the app booted from it. The + # app loads the newest committed one and deletes the older ones; nothing + # outside it is touched (MOB-408). + xcrun devicectl device info files ... --subdirectory Documents/otp/.versions + xcrun devicectl device copy to --device ... Documents/otp/.versions/ + xcrun devicectl device copy to --device ... Documents/otp/.versions//.mob_complete + xcrun devicectl device copy to --device ... Documents/mob_diag_beams_dir.txt + xcrun devicectl device process launch --device --terminate-existing + xcrun devicectl device copy from ... --source Documents/mob_diag_beams_dir.txt + # (an app on mob < 0.9.20, or not yet launched since its --native + # install: merge into Documents/otp/ instead) The BEAMs come from Mix's active build path (`Mix.Project.compile_path/0` and `Mix.Project.build_path/0`): the app and its runtime dependencies, never diff --git a/lib/mob_dev/deployer.ex b/lib/mob_dev/deployer.ex index aeda737..daabb6c 100644 --- a/lib/mob_dev/deployer.ex +++ b/lib/mob_dev/deployer.ex @@ -1128,13 +1128,20 @@ defmodule MobDev.Deployer do end end - # Physical iOS deploy: push BEAMs into the app's Documents container via - # `xcrun devicectl`. mob_beam.m (MOB_BUNDLE_OTP build) checks - # Documents/otp// at startup and prefers it over the read-only in-bundle - # copy, enabling fast deploys without a full Xcode rebuild. + # Physical iOS deploy: upload the BEAMs into the app's Documents with + # `xcrun devicectl device copy to`. mob_beam.m (MOB_BUNDLE_OTP build) loads + # them in preference to the read-only copy in the signed bundle, enabling + # fast deploys without a full Xcode rebuild. Where they go: `ios_code_plan/2`. # - # The merged staging dir is named so that devicectl's directory-copy - # semantics land the files at Documents/otp// on device. + # Never `--remove-existing-content`: whatever --destination says, it empties + # the app's whole data container (Documents, Library, tmp), taking the app's + # databases, sessions and settings with it (MOB-408). + @ios_versions_suffix ".versions" + @ios_complete_marker ".mob_complete" + # The mob runtime that reads .versions/ ships this header; see + # `mob_reads_ios_versions?/1`. + @ios_versions_header "ios/mob_code_versions.h" + @doc false @spec validate_ios_override(String.t(), String.t(), String.t()) :: :ok | {:error, String.t()} @@ -1169,8 +1176,115 @@ defmodule MobDev.Deployer do end @doc false - @spec ios_override_copy_args(String.t(), String.t(), String.t(), String.t()) :: [String.t()] - def ios_override_copy_args(udid, bundle, staging_dir, app) do + # Where a physical iPhone's BEAMs go on this deploy (MOB-408). + # + # `{:versioned, n}`: a fresh `Documents/otp/.versions/`, committed by + # copying `/.mob_complete` once the upload is verified. mob ≥ 0.9.20 loads + # the highest committed `n` (see mob's `ios/mob_code_versions.h`), so a copy + # that dies part-way leaves the previous version booting. `n` is above every + # entry already there, committed or not, so no upload lands on another's + # leftovers. + # + # `:legacy`: merge into `Documents/otp/`, the one directory older + # runtimes read. Nothing is deleted; a module removed from the project + # lingers there until the app is reinstalled or a newer runtime's first + # versioned boot deletes the directory. + # + # Versioned only when both sides say so: the project's mob ships the runtime + # that reads the layout (`mob_reads_ios_versions?/1`), and the device has + # `.versions/`, which that runtime creates at every boot (`listing` is + # `{:ok, entries}`). Either alone is often wrong: mob updated without + # `--native` leaves an old runtime on the phone, and a phone that ran a newer + # mob keeps the directory after a `--native` downgrade. Both together can + # still be wrong when the project's mob moves across 0.9.20 without + # `--native`, leaving the phone's runtime the other kind; + # `confirm_ios_code_loaded/4` turns that into a failed deploy rather than a + # phone quietly running old code. + @spec ios_code_plan(boolean(), {:ok, [String.t()]} | :absent) :: + {:versioned, String.t()} | :legacy + def ios_code_plan(true, {:ok, entries}), do: {:versioned, next_ios_code_version(entries)} + def ios_code_plan(_mob_reads_versions?, _listing), do: :legacy + + defp next_ios_code_version(entries) do + entries + |> Enum.filter(&Regex.match?(~r/\A\d{1,18}\z/, &1)) + |> Enum.map(&String.to_integer/1) + |> Enum.max(fn -> 0 end) + |> Kernel.+(1) + |> Integer.to_string() + |> String.pad_leading(10, "0") + end + + @doc false + # Whether the mob at `mob_dep_path` (the project's `:mob` dependency) has the + # iOS runtime that reads `.versions/`. + @spec mob_reads_ios_versions?(Path.t() | nil) :: boolean() + def mob_reads_ios_versions?(nil), do: false + + def mob_reads_ios_versions?(mob_dep_path), + do: File.regular?(Path.join(mob_dep_path, @ios_versions_header)) + + @doc false + # The app-container directory a plan uploads into. + @spec ios_code_dir(String.t(), {:versioned, String.t()} | :legacy) :: String.t() + def ios_code_dir(app, {:versioned, n}), do: "Documents/otp/#{app}#{@ios_versions_suffix}/#{n}" + def ios_code_dir(app, :legacy), do: "Documents/otp/#{app}" + + @doc false + @spec ios_versions_list_args(String.t(), String.t(), String.t(), String.t()) :: [String.t()] + def ios_versions_list_args(udid, bundle, app, json_path) do + [ + "devicectl", + "device", + "info", + "files", + "--device", + udid, + "--domain-type", + "appDataContainer", + "--domain-identifier", + bundle, + "--subdirectory", + "Documents/otp/#{app}#{@ios_versions_suffix}", + "--no-recurse", + "--json-output", + json_path, + "-q" + ] + end + + @doc false + # The entry names in `.versions/` from `ios_versions_list_args/4`'s JSON; + # `:absent` when devicectl can't list it, which is how it reports both a + # missing directory and an app that isn't installed (the copy that follows + # tells the two apart); `{:error, reason}` for anything else. + @spec parse_ios_versions_listing(String.t()) :: + {:ok, [String.t()]} | :absent | {:error, String.t()} + def parse_ios_versions_listing(json) do + case Jason.decode(json) do + {:ok, %{"result" => %{"files" => files}}} when is_list(files) -> + {:ok, for(%{"relativePath" => path} <- files, not String.contains?(path, "/"), do: path)} + + {:ok, %{"error" => %{"domain" => "CoreDevice.ActionError", "code" => 3}}} -> + :absent + + {:ok, %{"error" => error}} -> + {:error, "devicectl could not list the app's files: #{devicectl_error(error)}"} + + _ -> + {:error, "devicectl's file listing was not the JSON expected"} + end + end + + defp devicectl_error(%{"userInfo" => %{"NSLocalizedDescription" => %{"string" => s}}}), do: s + defp devicectl_error(%{"domain" => d, "code" => c}), do: "#{d} error #{c}" + defp devicectl_error(other), do: inspect(other) + + @doc false + # Copies `source` (a directory's contents, or one file) to `destination` in + # the app's container. A merge: nothing already on the device is removed. + @spec ios_code_copy_args(String.t(), String.t(), String.t(), String.t()) :: [String.t()] + def ios_code_copy_args(udid, bundle, source, destination) do [ "devicectl", "device", @@ -1183,17 +1297,16 @@ defmodule MobDev.Deployer do "--domain-identifier", bundle, "--source", - staging_dir, + source, "--destination", - "Documents/otp/#{app}", - "--remove-existing-content", - "true" + destination ] end @doc false - @spec ios_override_verify_args(String.t(), String.t(), String.t(), String.t()) :: [String.t()] - def ios_override_verify_args(udid, bundle, destination, app) do + @spec ios_code_verify_args(String.t(), String.t(), String.t(), String.t(), String.t()) :: + [String.t()] + def ios_code_verify_args(udid, bundle, remote_dir, app, destination) do [ "devicectl", "device", @@ -1206,7 +1319,7 @@ defmodule MobDev.Deployer do "--domain-identifier", bundle, "--source", - "Documents/otp/#{app}/#{app}.beam", + "#{remote_dir}/#{app}.beam", "--destination", destination ] @@ -1313,8 +1426,8 @@ defmodule MobDev.Deployer do File.write!(Path.join(staging_dir, "mob_beam_flags"), beam_flags) end - # Lands in Documents/otp// with the BEAMs; mob_beam.m reads it at - # boot when the launch env has no MOB_DIST_COOKIE (a tap on the icon). + # Lands next to the BEAMs; mob_beam.m reads it at boot when the launch + # env has no MOB_DIST_COOKIE (a tap on the icon). DistCookie.write_app_file!(staging_dir, dist_cookie) case validate_ios_override(compile_path, staging_dir, app) do @@ -1322,20 +1435,18 @@ defmodule MobDev.Deployer do {:error, reason} -> throw({:error, reason}) end - # devicectl copies the contents of --source into --destination. - # To land BEAMs at Documents/otp//, the destination must include - # the app subdirectory explicitly (staging_dir naming alone is not enough). - # From here the on-device override is being REPLACED, not merged - # (--remove-existing-content). Any failure past this point leaves it in - # an unknown state, and mob_beam.m prefers Documents/otp/ on mere - # directory existence — so there is no falling back to the signed - # bundle, and `xcrun devicectl device` has no delete verb to undo it. - # Record that we crossed the line so the error can say so. - Process.put(:mob_ios_override_replaced, true) + plan = plan_ios_code_upload(udid, bundle, app, staging_parent) + code_dir = ios_code_dir(app, plan) + + # devicectl copies the contents of --source into --destination, so the + # destination names the code directory itself. From here the device may + # hold a partial upload; record which, so an error can say what that + # means for the next launch. + Process.put(:mob_ios_code_upload, {plan, code_dir}) case System.cmd( "xcrun", - ios_override_copy_args(udid, bundle, staging_dir, app), + ios_code_copy_args(udid, bundle, staging_dir, code_dir), stderr_to_stdout: true ) do {_, 0} -> @@ -1380,7 +1491,7 @@ defmodule MobDev.Deployer do case System.cmd( "xcrun", - ios_override_verify_args(udid, bundle, received_bootstrap, app), + ios_code_verify_args(udid, bundle, code_dir, app, received_bootstrap), stderr_to_stdout: true ) do {_, 0} -> @@ -1393,11 +1504,20 @@ defmodule MobDev.Deployer do throw({:error, "physical iOS bootstrap verification failed: #{out}"}) end + commit_ios_code_upload(udid, bundle, plan, code_dir, staging_parent) + # Committed: from here a failure leaves no partial upload behind. + Process.delete(:mob_ios_code_upload) + if restart do - IOS.restart_app_physical(udid, bundle, dist_cookie: Atom.to_string(dist_cookie)) + clear_ios_loaded_dir(udid, bundle, staging_parent) + + case IOS.restart_app_physical(udid, bundle, dist_cookie: Atom.to_string(dist_cookie)) do + {_, 0} -> confirm_ios_code_loaded(udid, bundle, code_dir, staging_parent) + # Not launched (a locked phone, say): nothing booted to confirm. + _ -> :ok + end end - Process.delete(:mob_ios_override_replaced) {:ok, device} catch {:skipped, reason} -> @@ -1406,41 +1526,177 @@ defmodule MobDev.Deployer do {:error, reason} -> finalize_ios_override_result({:error, reason}, app) after - Process.delete(:mob_ios_override_replaced) + Process.delete(:mob_ios_code_upload) File.rm_rf!(staging_parent) end end + # Lists .versions/ only when the project's mob could read it; an older + # mob means :legacy whatever the device holds. + defp plan_ios_code_upload(udid, bundle, app, staging_parent) do + if mob_reads_ios_versions?(MobDev.MobDirCheck.dep_path()) do + json_path = Path.join(staging_parent, "versions.json") + args = ios_versions_list_args(udid, bundle, app, json_path) + # Exits non-zero for a missing directory too; the JSON says which. + {out, _status} = System.cmd("xcrun", args, stderr_to_stdout: true) + + listing = + case File.read(json_path) do + {:ok, json} -> parse_ios_versions_listing(json) + {:error, _} -> {:error, "devicectl could not list the app's files: #{out}"} + end + + case listing do + {:error, reason} -> throw({:error, reason}) + listing -> ios_code_plan(true, listing) + end + else + ios_code_plan(false, :absent) + end + end + + # A versioned upload counts only once its marker is there: mob_beam.m never + # loads a version without one. Written last, after the bootstrap check. + defp commit_ios_code_upload(_udid, _bundle, :legacy, _code_dir, _staging_parent), do: :ok + + defp commit_ios_code_upload(udid, bundle, {:versioned, n}, code_dir, staging_parent) do + marker = Path.join(staging_parent, "mob_complete") + File.write!(marker, n <> "\n") + destination = "#{code_dir}/#{@ios_complete_marker}" + + case System.cmd("xcrun", ios_code_copy_args(udid, bundle, marker, destination), + stderr_to_stdout: true + ) do + {_, 0} -> :ok + {out, _} -> throw({:error, "devicectl could not write #{destination}: #{out}"}) + end + end + + # The upload is on the phone, but the runtime picks what it loads, and a + # wrong guess in ios_code_plan/2 (an app whose native runtime and :mob + # dependency differ, say) would have it boot other code while the deploy + # reports success. Every mob runtime since 0.6.2 writes the directory it + # chose to Documents/mob_diag_beams_dir.txt at boot, so wait for it to name + # this one. + @ios_boot_confirm_ms 20_000 + @ios_loaded_dir_file "Documents/mob_diag_beams_dir.txt" + @ios_loaded_dir_placeholder "(relaunched by mix mob.deploy, not booted yet)\n" + + # Overwrites the previous boot's answer before the relaunch, so only the new + # boot can satisfy confirm_ios_code_loaded/4: a legacy deploy expects the + # same path every time. + defp clear_ios_loaded_dir(udid, bundle, staging_parent) do + placeholder = Path.join(staging_parent, "mob_diag_placeholder.txt") + File.write!(placeholder, @ios_loaded_dir_placeholder) + args = ios_code_copy_args(udid, bundle, placeholder, @ios_loaded_dir_file) + + case System.cmd("xcrun", args, stderr_to_stdout: true) do + {_, 0} -> :ok + {out, _} -> throw({:error, "devicectl could not reset #{@ios_loaded_dir_file}: #{out}"}) + end + end + + defp confirm_ios_code_loaded(udid, bundle, code_dir, staging_parent) do + destination = Path.join(staging_parent, "mob_diag_beams_dir.txt") + args = ios_loaded_dir_args(udid, bundle, destination) + deadline = System.monotonic_time(:millisecond) + @ios_boot_confirm_ms + await_ios_code_loaded(args, destination, code_dir, deadline) + end + + defp await_ios_code_loaded(args, destination, code_dir, deadline) do + File.rm(destination) + System.cmd("xcrun", args, stderr_to_stdout: true) + loaded = with {:ok, diag} <- File.read(destination), do: diag, else: (_ -> nil) + + cond do + ios_loaded?(loaded, code_dir) -> + :ok + + System.monotonic_time(:millisecond) >= deadline -> + throw({:error, ios_not_loaded_message(loaded, code_dir)}) + + true -> + Process.sleep(500) + await_ios_code_loaded(args, destination, code_dir, deadline) + end + end + + @doc false + @spec ios_loaded_dir_args(String.t(), String.t(), String.t()) :: [String.t()] + def ios_loaded_dir_args(udid, bundle, destination) do + [ + "devicectl", + "device", + "copy", + "from", + "--device", + udid, + "--domain-type", + "appDataContainer", + "--domain-identifier", + bundle, + "--source", + @ios_loaded_dir_file, + "--destination", + destination + ] + end + + @doc false + # Whether the runtime's beams-dir diagnostic (an absolute path on the + # phone, one line) names `code_dir`, a path relative to the app container. + @spec ios_loaded?(String.t() | nil, String.t()) :: boolean() + def ios_loaded?(nil, _code_dir), do: false + + def ios_loaded?(diag, code_dir) do + case String.split(diag, "\n", parts: 2) do + [path | _] -> String.ends_with?(String.trim_trailing(path, "/"), "/" <> code_dir) + _ -> false + end + end + + defp ios_not_loaded_message(loaded, code_dir) + when loaded in [nil, "", @ios_loaded_dir_placeholder] do + "the app was relaunched but didn't report which BEAMs it loaded within " <> + "#{div(@ios_boot_confirm_ms, 1000)} s (#{@ios_loaded_dir_file}), so #{code_dir} " <> + "may not be running. Open the app and check the device log." + end + + defp ios_not_loaded_message(loaded, code_dir) do + "the app relaunched from #{String.trim(loaded)}, not #{code_dir}, which this " <> + "deploy wrote, so it still runs older code. The app's native runtime and the " <> + "project's :mob dependency don't match; run `mix mob.deploy --native`." + end + @doc false @spec finalize_ios_override_result({:skipped | :error, String.t()}, String.t()) :: {:skipped | :error, String.t()} def finalize_ios_override_result({:skipped, reason}, _app), do: {:skipped, reason} def finalize_ios_override_result({:error, reason}, app), - do: {:error, annotate_override_state(reason, app)} + do: {:error, reason <> ios_partial_upload_note(Process.get(:mob_ios_code_upload), app)} - # If the destructive copy had already begun, the device is now in a worse - # state than we found it: Documents/otp/ exists but is incomplete, and - # mob_beam.m selects that override on directory existence alone. The app will - # boot from it — and fail — on the next launch from Springboard, even though - # we correctly skipped the restart. - # - # There is no clean rollback. `xcrun devicectl device` has no delete verb, and - # copying an EMPTY directory does not help: the directory still exists, so it - # is still preferred, and the app would then have no BEAMs at all. So say so - # plainly and give the two recoveries that do work. - defp annotate_override_state(reason, app) do - if Process.get(:mob_ios_override_replaced) do - reason <> - "\n\nThe on-device override at Documents/otp/#{app} was already being " <> - "replaced when this failed, so it is now incomplete. The app prefers that " <> - "directory over its signed bundle, so the next launch will fail even " <> - "though it was not restarted.\n" <> - "Recover with a successful `mix mob.deploy` (or `--native`), or delete " <> - "and reinstall the app." - else - reason - end + @doc false + # What a failure after the upload began leaves on the phone, for the error. + # `upload` is the `{plan, code_dir}` the deploy had started, or nil. + @spec ios_partial_upload_note( + {{:versioned, String.t()} | :legacy, String.t()} | nil, + String.t() + ) :: String.t() + def ios_partial_upload_note(nil, _app), do: "" + + def ios_partial_upload_note({{:versioned, _n}, code_dir}, _app) do + "\n\nNothing else on the phone was changed, and it keeps running the previous " <> + "code: the app loads #{code_dir} only once #{@ios_complete_marker} is in it " <> + "(the deploy's last step), and deletes it after the next deploy that boots." + end + + def ios_partial_upload_note({:legacy, code_dir}, app) do + "\n\nThe copy into #{code_dir} was under way when this failed, so it may now " <> + "mix new and old BEAMs, which the app loads at its next launch. Recover with " <> + "a successful `mix mob.deploy`. Deploys go to #{app}#{@ios_versions_suffix}/, " <> + "where an interrupted one leaves the previous code running, once the app " <> + "runs mob ≥ 0.9.20 (`mix mob.deploy --native`) and has been launched." end # ── iOS WiFi UDID resolution ────────────────────────────────────────────────── @@ -1646,10 +1902,11 @@ defmodule MobDev.Deployer do # back to dist-only in the deployer"), and a LAN-only device has no # `devicectl` route at all — the first version of this change turned that # documented fallback into a hard exit 1 on a deploy that had previously - # succeeded. Even with USB attached, the write is an - # `xcrun devicectl ... --remove-existing-content` replace with no undo, so a - # cable knock mid-copy leaves the app unbootable. A hot load could never - # damage a device; making it able to, silently, is not a fix. + # succeeded. Even with USB attached, the write could damage the app: for a + # runtime older than mob 0.9.20 it merges into the one directory the app + # boots from, so a cable knock mid-copy can leave it unbootable (and before + # MOB-408 it emptied the app's whole data container). A hot load could + # never damage a device; making it able to, silently, is not a fix. # # The documented physical-iOS dist workflow is USB *unplugged* anyway, which # is exactly the state in which the write cannot run. diff --git a/test/mob_dev/deployer_dist_persist_test.exs b/test/mob_dev/deployer_dist_persist_test.exs index b88bd4d..931ebd7 100644 --- a/test/mob_dev/deployer_dist_persist_test.exs +++ b/test/mob_dev/deployer_dist_persist_test.exs @@ -31,13 +31,13 @@ defmodule MobDev.DeployerDistPersistTest do end test "a physical iPhone is not" do - # Two independent reasons, either sufficient. It is discovered by probing - # EPMD across the LAN, and a LAN-only device has no `devicectl` route at - # all — the first version of this fix turned that documented fallback - # into a hard exit 1 on a deploy that previously succeeded. And even over - # USB the write is a `--remove-existing-content` replace with no undo, so - # a cable knock mid-copy leaves the app unbootable. A hot load could - # never damage a device; making it able to, silently, is not a fix. + # It is discovered by probing EPMD across the LAN, and a LAN-only device + # has no `devicectl` route at all — the first version of this fix turned + # that documented fallback into a hard exit 1 on a deploy that previously + # succeeded. And even over USB, for an app on mob < 0.9.20 the write + # merges into the one directory it boots from, so a cable knock mid-copy + # can leave it unbootable. A hot load could never damage a device; making + # it able to, silently, is not a fix. refute Deployer.persistable?(device(platform: :ios, type: :physical)) end end diff --git a/test/mob_dev/deployer_test.exs b/test/mob_dev/deployer_test.exs index 69bb402..8772be1 100644 --- a/test/mob_dev/deployer_test.exs +++ b/test/mob_dev/deployer_test.exs @@ -17,9 +17,9 @@ defmodule MobDev.DeployerTest do end describe "physical iOS override" do - test "app-not-installed skips never claim the override is incomplete" do - Process.put(:mob_ios_override_replaced, true) - on_exit(fn -> Process.delete(:mob_ios_override_replaced) end) + test "app-not-installed skips carry no partial-upload note" do + Process.put(:mob_ios_code_upload, {:legacy, "Documents/otp/sample_app"}) + on_exit(fn -> Process.delete(:mob_ios_code_upload) end) assert {:skipped, "App is not installed"} = Deployer.finalize_ios_override_result( @@ -28,15 +28,31 @@ defmodule MobDev.DeployerTest do ) end - test "real copy failures still explain an incomplete override" do - Process.put(:mob_ios_override_replaced, true) - on_exit(fn -> Process.delete(:mob_ios_override_replaced) end) + test "an error before the upload began is reported as is" do + Process.delete(:mob_ios_code_upload) + + assert {:error, "listing failed"} = + Deployer.finalize_ios_override_result({:error, "listing failed"}, "sample_app") + end + + test "an error after the upload began says which directory it left behind" do + dir = "Documents/otp/sample_app.versions/0000000007" + Process.put(:mob_ios_code_upload, {{:versioned, "0000000007"}, dir}) + on_exit(fn -> Process.delete(:mob_ios_code_upload) end) + + assert {:error, message} = + Deployer.finalize_ios_override_result({:error, "copy failed"}, "sample_app") + + assert String.starts_with?(message, "copy failed") + assert message =~ dir + + Process.put(:mob_ios_code_upload, {:legacy, "Documents/otp/sample_app"}) assert {:error, message} = Deployer.finalize_ios_override_result({:error, "copy failed"}, "sample_app") - assert message =~ "copy failed" - assert message =~ "now incomplete" + assert String.starts_with?(message, "copy failed") + assert message =~ "Documents/otp/sample_app" end test "requires staged bootstrap bytes to match the active compile output" do @@ -79,8 +95,10 @@ defmodule MobDev.DeployerTest do Deployer.validate_ios_override(compile_path, staging_dir, "sample_app") end - test "copy command replaces the exact override directory" do - args = Deployer.ios_override_copy_args("device-id", "com.example.app", "/tmp/app", "app") + # MOB-408: --remove-existing-content empties the app's whole data + # container (Documents, Library, tmp), whatever --destination names. + test "the copy merges into its destination and never removes existing content" do + args = Deployer.ios_code_copy_args("device-id", "com.example.app", "/tmp/app", "dest/dir") assert args == [ "devicectl", @@ -96,18 +114,126 @@ defmodule MobDev.DeployerTest do "--source", "/tmp/app", "--destination", - "Documents/otp/app", - "--remove-existing-content", - "true" + "dest/dir" ] end - test "verification command retrieves the exact remote bootstrap" do - assert Deployer.ios_override_verify_args( + # What every mob runtime since 0.6.2 writes to Documents/mob_diag_beams_dir.txt + # at boot: the absolute path of the directory it loads. + test "the deploy is confirmed only when the app booted from the directory it wrote" do + container = "/var/mobile/Containers/Data/Application/9F50A0EA" + versioned = "Documents/otp/app.versions/0000000008" + + assert Deployer.ios_loaded?("#{container}/#{versioned}\n", versioned) + assert Deployer.ios_loaded?("#{container}/Documents/otp/app\n", "Documents/otp/app") + + # An older committed version, the legacy override, or the bundle. + refute Deployer.ios_loaded?("#{container}/Documents/otp/app.versions/0000000007", versioned) + refute Deployer.ios_loaded?("#{container}/Documents/otp/app\n", versioned) + refute Deployer.ios_loaded?("#{container}/#{versioned}\n", "Documents/otp/app") + refute Deployer.ios_loaded?("/private/var/containers/Bundle/D.app/otp/app", versioned) + # Another app whose name ends in this one's. + refute Deployer.ios_loaded?("#{container}/Documents/otp/myapp\n", "Documents/otp/app") + # No diagnostic yet. + refute Deployer.ios_loaded?(nil, versioned) + refute Deployer.ios_loaded?("", versioned) + end + + test "a versioned deploy goes above every existing entry, committed or not" do + # An interrupted upload (no marker) still takes its number: reusing it + # would merge into its leftovers. + assert {:versioned, "0000000004"} = + Deployer.ios_code_plan( + true, + {:ok, ["0000000001", "0000000003", "notes", "0000000002", "12x"]} + ) + + assert {:versioned, "0000000001"} = Deployer.ios_code_plan(true, {:ok, []}) + assert {:versioned, "0000000010"} = Deployer.ios_code_plan(true, {:ok, ["9"]}) + end + + test "versioned only when both the project's mob and the phone's runtime read it" do + # The phone never created .versions/: its runtime predates the + # layout (mob updated without --native), or hasn't booted yet. + assert :legacy = Deployer.ios_code_plan(true, :absent) + # The project's mob predates it: the runtime about to run is old even if + # the phone kept .versions/ from a newer one. + assert :legacy = Deployer.ios_code_plan(false, {:ok, ["0000000001"]}) + assert :legacy = Deployer.ios_code_plan(false, :absent) + end + + test "each plan uploads into its own directory" do + assert Deployer.ios_code_dir("app", {:versioned, "0000000002"}) == + "Documents/otp/app.versions/0000000002" + + assert Deployer.ios_code_dir("app", :legacy) == "Documents/otp/app" + end + + test "the project's mob reads .versions/ when it ships the runtime header" do + dir = Path.join(System.tmp_dir!(), "mob_dep_#{System.unique_integer([:positive])}") + File.mkdir_p!(Path.join(dir, "ios")) + on_exit(fn -> File.rm_rf!(dir) end) + + refute Deployer.mob_reads_ios_versions?(dir) + File.write!(Path.join(dir, "ios/mob_code_versions.h"), "") + assert Deployer.mob_reads_ios_versions?(dir) + refute Deployer.mob_reads_ios_versions?(nil) + end + + test "the listing lists .versions/ itself, one level, as JSON to a file" do + args = Deployer.ios_versions_list_args("device-id", "com.example.app", "app", "/tmp/l.json") + + assert ["devicectl", "device", "info", "files" | _] = args + + assert ["--subdirectory", "Documents/otp/app.versions" | _] = + drop_until(args, "--subdirectory") + + assert "--no-recurse" in args + assert ["--json-output", "/tmp/l.json" | _] = drop_until(args, "--json-output") + + assert ["--domain-identifier", "com.example.app" | _] = + drop_until(args, "--domain-identifier") + end + + # Shapes captured from devicectl 642.16 against an iPhone (MOB-408). + test "a listing yields the directory's entry names" do + json = ~s({"info": {"outcome": "success"}, "result": {"files": [ + {"name": "0000000001", "relativePath": "0000000001", + "resources": {"isDirectory": true}}, + {"name": ".mob_complete", "relativePath": "0000000001/.mob_complete", + "resources": {"isDirectory": false}}, + {"name": "0000000002", "relativePath": "0000000002", + "resources": {"isDirectory": true}}]}}) + + assert {:ok, ["0000000001", "0000000002"]} = Deployer.parse_ios_versions_listing(json) + assert {:ok, []} = Deployer.parse_ios_versions_listing(~s({"result": {"files": []}})) + end + + test "a directory devicectl can't list is absent; other failures are errors" do + # What a missing .versions/ (and an app that isn't installed) gives. + missing = ~s({"error": {"code": 3, "domain": "CoreDevice.ActionError", "userInfo": { + "NSDebugDescription": {"string": "The system failed to get a list of files"}}}, + "info": {"outcome": "failed"}}) + + assert :absent = Deployer.parse_ios_versions_listing(missing) + + no_device = ~s({"error": {"code": 1000, "domain": "com.apple.dt.CoreDeviceError", + "userInfo": {"NSLocalizedDescription": {"string": "The specified device was not found."}}}, + "info": {"outcome": "failed"}}) + + assert {:error, message} = Deployer.parse_ios_versions_listing(no_device) + assert message =~ "The specified device was not found." + + assert {:error, _} = Deployer.parse_ios_versions_listing("ERROR: not json") + end + + test "verification reads back the bootstrap from the directory just written" do + assert Deployer.ios_code_verify_args( "device-id", "com.example.app", - "/tmp/received.beam", - "app" + "Documents/otp/app.versions/0000000002", + "app", + "/tmp/received.beam" ) == [ "devicectl", "device", @@ -120,7 +246,7 @@ defmodule MobDev.DeployerTest do "--domain-identifier", "com.example.app", "--source", - "Documents/otp/app/app.beam", + "Documents/otp/app.versions/0000000002/app.beam", "--destination", "/tmp/received.beam" ] @@ -539,4 +665,6 @@ defmodule MobDev.DeployerTest do ] end end + + defp drop_until(list, item), do: Enum.drop_while(list, &(&1 != item)) end