[thymikee]

Summary

Android now runs the iOS command suite against the fixture app. One shared suite.ts holds the steps, and platform deltas are data (only, on.android). One harness, suite-harness.ts, owns rounds, sessions, and daemon restarts. The Settings tour is removed. 30 files changed.

• Platform-only steps: Android has hardware-button, push, and telephony. iOS has action-button, prepare, clipboard, and home-reads/snapshot-forced-tree.
• Heavy feed: raw depth must be ≥56. snapshot-raw-one-shot restarts the daemon with AGENT_DEVICE_ANDROID_SNAPSHOT_HELPER_SESSION=0 and must be served by instrumentation (ADR 0002).
• Profiles: each Android helper capture timer records its typed transport, and the session timer also records answered. profile.ts reads these fields and no longer infers the transport from event order.
• Nightly: Android runs on schedule and uploads summary and raw artifacts. perf:compare gains --platform.

Validation

• At 1fddbe9, heavy-feed ran twice on an API 35 emulator (3 rounds + 1 warmup), with 0 failed samples. In run 2, the session steps were served by persistent-session ×3 and the one-shot step by instrumentation ×3. In every sample, the profile matches the response.
• At 70faef4, Android heavy-feed,automation and iOS home-reads,heavy-feed ran clean, apart from the 3 known iOS heavy-feed gaps.
• vitest scripts/perf passed 45/45. The platform-android snapshot* tests passed 190/190.
• pnpm check:affected --run --head 1fddbe9d0f passed (exactCommit: true, 37 passed, 22 GitHub-authoritative skips).
• Risk: the nightly Android route runs only on schedule or workflow_dispatch.

🤖 Generated with Claude Code

[github-actions]

Size Report

Metric	Base	Current	Diff
Installed (including dependencies)	5.55 MB	5.55 MB	+253 B
Package (unpacked)	5.55 MB	5.55 MB	+253 B
Package (download)	1.68 MB	1.68 MB	+65 B

Startup median (7 runs, lower is better):

Scenario	Base	Current	Diff
CLI --version	41.0 ms	41.3 ms	+0.3 ms
CLI --help	84.9 ms	83.8 ms	-1.1 ms

[thymikee]

The code at 4f18a1a looks ready, and I found nothing that needs to change. All 11 checks pass, but the perf nightly workflow this PR changes is not a PR check, so only the cited workflow_dispatch runs exercise the Android suite. I did not open those runs or their artifacts, so the 104-step and 0-failure results are the author's claim. The PR has merge conflicts with its base branch: the head carries pre-rebase copies of the base commits. Please rebase onto the current perf/ios-command-suite branch so the PR shows only its two commits, then re-run the checks. Base PR #3885 must land first. I reviewed the logical patch from 558da5c to the head, so any conflicts that appear after the rebase are not reviewed. Not blocking, and you can take or leave these: scripts/perf/profile.ts:112 counts a skipped persistent-session capture as a failure when snapshot-raw-one-shot turns the session off on purpose, so it could count that capture without a failure, and scripts/perf/suite-harness.ts:48 branches on platform in three places, which one per-platform record could replace.

[thymikee]

New head: 181d526. #3884 merged, and the maintainer rebased #3885 onto main. This layer takes the new #3885 head (8958314) through a merge commit, so nothing was force-pushed.

Merge (88d7d73)

• This layer's two commits were replayed onto 8958314, and the result was recorded as a merge of the old and new heads.
• git diff 8958314a16 88d7d73c0c matches the layer's old diff (558da5cdbb..4f18a1a4f1) except for the ported feat(perf): time every iOS command against the fixture app #3885 fixes. Those are the pinned-tree step, shared host readers, phase constants, summary.fixtures.ts, and the nightly header.
• In the renamed suite.ts and suite-harness.ts:
  • The iOS pinned capture is home-reads/snapshot-forced-tree (only: 'ios'). It is a regular capture, and it must be served by tree with requested-backend. See the feat(perf): time every iOS command against the fixture app #3885 comment: the old raw heavy-feed/snapshot-raw-forced-tree row was served by private AX.
  • heavy-feed/snapshot-raw-one-shot (only: 'android') keeps expect: { maxDepthAtLeast: 56, transport: 'instrumentation' }. A sample that any other transport served fails with expectation_failed. suite.test.ts covers that case and the unknown-transport case.
• metadata.ts uses the shared scripts/lib/bench-host.ts readers (revision, toolchain, simulator). It keeps one adb-based emulator reader, because no other owner reads Android target identity. On Linux, readToolchain returns unavailable for Xcode and does not throw.

Follow-ups

• 70faef4: platform-android exports ANDROID_SNAPSHOT_HELPER_SESSION_CAPTURE_PHASE, ANDROID_SNAPSHOT_HELPER_CAPTURE_PHASE, and ANDROID_TOUCH_HELPER_GESTURE_PHASE from their emitting modules through /mechanics. profile.ts and profile.test.ts import them, and check:production-exports is clean. The README's snapshot-path section and parity table name the pinned capture.
• 181d526: fixes from an independent review of 8958314a16..70faef463c, which found nothing blocking.
  • stopStuckDaemon now logs the failed restore when that, not a timed-out request, caused the restart.
  • The README's local compare example points at .tmp/perf/ios and adds the Android --platform android --to .tmp/perf/android line.

Device evidence at 70faef4

The commit after it changes only a log string and the README.

• Android: emulator PerfStackWorker_API_35 (API 35), --scenarios heavy-feed,automation --rounds 3 --warmup 1. Result: 34 steps, 0 failed measured samples.

heavy-feed/snapshot-raw          (persistent-session) ×3: depth 60, 193 nodes, daemon 152 / 135 / 412 ms
heavy-feed/snapshot-raw-one-shot (instrumentation) ×3:    depth 60, 231 nodes, daemon 1504 / 1263 / 775 ms
| snapshot-raw-one-shot | 0 | 1 / 1248 | 1248 | – | 0 | 0 | android-helper/instrumentation×3 | snapshot_capture 1255, android_snapshot_helper_capture 1248, exec_command 1248, … |

The only failure was in the dropped warmup: automation/record-stop hit xcrun swiftc building the video validator into a fresh Swift cache, which timed out at 30 s on a loaded macOS host. The Linux nightly has no xcrun. It did not recur.

• iOS: home-reads,heavy-feed through the merged harness. home-reads/snapshot-forced-tree returned tree/healthy/requested-backend ×3 (execMode: sdk), with a median of 271 ms. The only failures were the known heavy-feed gaps (longpress-post, swipe, back), which match the base.

Validation

• vitest scripts/perf scripts/lib: 44 tests passed. check:production-exports and lint were clean.
• pnpm check:affected --run --head 181d5260bb: queued behind other workers on the shared host lock. I will add the receipt to the PR body.

Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com

[thymikee]

Head is now 50838e4: a merge of #3885's 9e9ccc3, which keeps the daemon_request phase literal so the fallow audit has no unused root export.

pnpm check:affected --run --head 50838e4397 passed: exactCommit: true, FAIL-OPEN full set, 36 checks passed, and 25 GitHub-authoritative checks were skipped. The device evidence above is from 70faef4. The later commits change only a log string, the README, and that one import.

One observation from the earlier failed gate run, unrelated to this PR: test/integration/provider-scenarios/composite-deadline.test.ts ("routed batch install reports its deadline…") fails deterministically when android/ime-helper/dist exists in the checkout, such as after pnpm build:android. It fails with Unscripted Android provider call: shell settings get secure default_input_method. With that directory moved away, the same commit passes, and it also passes in a checkout that never built the helpers. The scenario depends on local build artifacts. That is a test-isolation gap on main, not something this layer changes.

Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com

[thymikee]

Head is now 50502ec. It merges #3885's aed5aa7, which has main with #3892 and drops the logs-stop workaround. In the shared suite.ts, logs-stop is now plain step('logs-stop', C.logs, ['logs', 'stop']). It never had an on.android variant, so the removal applies to both platforms. The suite still restarts the daemon when a request outlives its timeout.

Device check at 50502ec on my own API 35 emulator, with dirty: false. Command: pnpm perf --platform android --scenarios automation --rounds 3 --warmup 1. Every automation step passed, with no daemon_stuck.

automation/logs-start  3/3 ok  daemon 64 / 56 / 54 ms
automation/logs-stop   3/3 ok  daemon 27 / 25 / 22 ms

The same scenario on iOS at the parent head is clean too; see #3885.

pnpm check:affected --run --head 50502eced9 passed: exactCommit: true, 36 checks passed, and 25 GitHub-authoritative checks were skipped.

Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com

[thymikee]

The conflict from my earlier review (#3902 (comment)) is fixed at 50502ec, and I found no code problems in the two new commits since 181d526. All 11 checks pass at this commit. I did not run a device. The Android emulator result comes from your comment, so that evidence is taken as reported. The PR still waits for its base #3885 (perf/ios-command-suite) to land first.

One design question on profile.ts that I would like answered before merge: it works out that the persistent session did not answer from event order (a clean session-capture event, then a one-shot capture, tracked in sessionAnswered). It also re-spells the transport names that platform-android already owns in AndroidSnapshotHelperTransport (snapshot-helper-types.ts:3). Could captureAndroidUiHierarchyFromHelper record this fact at its source instead, for example answered: Boolean(sessionCapture) (or the answering transport) in the session-capture timer's data? Then profile.ts would read one field per event and could drop the sessionAnswered state and the hard-coded producer strings.

[thymikee]

I merged main after #3885 squash-merged (no force-push). main's scripts/perf and perf-nightly.yml are byte-identical to #3885's final head (aed5aa7), so every conflict kept this branch's side. This PR's diff against main is unchanged: 29 files, +982/−780. vitest run scripts/perf passes 44/44, and lint and format pass.

[thymikee]

I found no code problems at fdacf18. The Android timing run is the same logical patch as before, so the results you reported at 70faef4 and 50502ec still apply.

All 16 PR checks pass and I know of no conflicts. perf-nightly.yml is not a PR check, so CI does not run the Android suite route. Only your workflow_dispatch and local runs cover it. I did not run a device, open the workflow_dispatch artifacts, or rerun vitest, lint or format.

The design question in #3902 (comment) is still open. Can the answering transport be recorded where it is produced, instead of being inferred in sessionAnswered? Please answer it, or say why inference is the better owner, before merge.

[thymikee]

Head is now 1fddbe9. It answers the design question from #3902 (comment): yes, the helper now records the transport at its source, and profile.ts no longer infers it. Nothing was force-pushed. The branch merged main (f3d2619) first.

Change

• Producer (packages/platform-android/src/snapshot.ts, captureAndroidUiHierarchyFromHelper): each helper capture timer records transport, typed as AndroidSnapshotHelperTransport.
  • The session timer records transport: 'persistent-session' and answered: sessionCapture !== undefined. Its data is evaluated when the timer settles, so the field reflects the actual result.
  • The one-shot timer records transport: 'instrumentation'.
  • The phase constants' docs state the contract.
• Consumer (scripts/perf/profile.ts): both helper phases share one branch. The producer is android-helper/${data.transport}, and data.answered === false counts as a failure of that producer. The sessionAnswered state and the ANDROID_SESSION_PRODUCER / ANDROID_ONE_SHOT_PRODUCER strings are deleted. profile.ts imports nothing new, because it reads the producer name from the event.

Why I kept the answered flag instead of recording only the answering transport

captureAndroidSnapshotWithHelperSession returns undefined both when the session is off on purpose and when it failed and retired itself, for example after malformed output or a timeout. In both cases its timer settles at info. If only the answering transport were recorded, that timer would carry no producer, and a session that spent 5 s before falling back would lose its time from snapshotSourceMs. With transport on every timer and answered on the session timer, each event names its own producer, and the profile needs no state between events.

The old inference was also wrong in two cases that the new tests pin:

1. A content re-capture after a session answer was counted as a session failure.
2. A request that ended before any one-shot capture counted an unanswered session as a success.

Tests (each fails without its change)

• packages/platform-android/src/__tests__/snapshot-helper-capture-timers.test.ts (new): drives snapshotAndroid through the fake persistent helper and reads the request log.
  • When the session answers: persistent-session / answered: true.
  • After a malformed session response: persistent-session / answered: false, then instrumentation.
  • It fails against the previous snapshot.ts.
  • It is a new file because snapshot.test.ts is at its lint max-lines limit.
• scripts/perf/profile.test.ts: the fixtures now carry the producer fields. A new test checks that a session answer followed by a one-shot capture keeps the session clean, and that an unanswered session with no one-shot capture after it is still counted. It fails against the previous profile.ts.

Evidence

• vitest run scripts/perf: 45/45 passed. All 22 platform-android snapshot* test files passed: 190 tests, including the new file.
• lint, format:check, typecheck, and check:production-exports are clean.
• Device: my own emulator review3902_API_35 (API 35, emulator-5618). Command: pnpm perf --platform android --serial emulator-5618 --scenarios heavy-feed --rounds 3 --warmup 1. Both runs had 0 failed samples. Run 2 table, with the response transport per sample in each step:

heavy-feed/snapshot-raw          persistent-session | persistent-session | persistent-session
heavy-feed/snapshot-i-scrolled   persistent-session | persistent-session | persistent-session
heavy-feed/snapshot-raw-one-shot instrumentation    | instrumentation    | instrumentation

Each step's profile producers agree with the response: android-helper/persistent-session for the session steps, and android-helper/instrumentation×3 for the forced one-shot step. The raw request logs carry the fields, for example:

android_snapshot_helper_session_capture … "transport":"persistent-session","answered":false
android_snapshot_helper_capture 507 ms   … "transport":"instrumentation"

• Observation from run 1 (not caused by this PR, and it did not reproduce): in the first measured round, the session returned no capture for every request. It settled at 0 ms with no adb call and no warning. That round came right after the warmup's forced one-shot step restored the daemon, so one snapshot-raw sample was answered by instrumentation. The response and the new profile fields both reported this correctly. Run 2 was clean, and a 5× reproduction of the harness's stop → SESSION=0 open → stop → open sequence restored persistent-session every time. Take or leave: heavy-feed/snapshot-raw could carry expect: { transport: 'persistent-session' }, so that a night where the session silently stays off fails as expectation_failed instead of mixing one-shot timings into the session baseline.
• pnpm check:affected --run --head 1fddbe9d0f passed: exactCommit: true, 37 checks passed, and 22 GitHub-authoritative checks were skipped. An earlier run of the gate at the same head, made while the emulator and perf suite loaded the host, timed out two tests in src/daemon-entrypoint.test.ts (5 s timeouts). That file passes 7/7 alone, and the re-run on a quiet host passed.
• CI at 1fddbe9: every check passed. The Apple Smoke Tests job shows as pending only because GitHub has not closed its record: all of its steps, including Complete job, succeeded, and its run concluded success.

Co-Authored-By: Claude Opus 5.5 noreply@anthropic.com

[thymikee]

This PR is ready at 1fddbe9. The earlier design question is answered: each snapshot-helper capture timer now records its transport and whether the session answered, and the emulator run you reported reaches the changed diagnostics on snapshot-raw, snapshot-i-scrolled and the forced one-shot step. Not blocking, take or leave: in packages/platform-android/src/snapshot.ts (https://github.com/callstack/agent-device/blob/1fddbe9/packages/platform-android/src/snapshot.ts#L482) sessionCapture is assigned twice, once inside the timer callback (needed so the data thunk sees it) and again from the withDiagnosticTimer return value, which is the same object, so you can call await withDiagnosticTimer(...) as a statement and keep only the inner assignment. I did not rerun the test, lint or typecheck gates or a device; I rely on your report that they pass at this commit. Only Smoke Tests is not passing, and it is still queued rather than failed. It does not overlap with this diff, since the Swift files in the delta come from merging main, so it needs to report completed before merge. There are no conflicts.

[github-actions]

PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-10-22 15:40 UTC