fix(settle): resolve the default quiet window from the capture it settles

[thymikee]

Part of #3470's follow-up work; does not close it. #3482 and #3512 have both merged, so this now targets main directly and carries only its own 8 files.

Summary

--settle asked every capture the same 500 ms quiet window. The iOS simulator snapshot bridge answers well inside that, so the loop spends a whole extra capture holding a window the tree has already satisfied — and the window is paid twice per settled command (the settle's final window, then again as the next command's baseline capture).

The loop now prices the window from the capture it performs, not the one the action was performed against: an unrequested --settle resolves 300 ms when that read came from the simulator snapshot bridge, and 500 ms for every other read — the Apple runner's own tree read, an Android capture from either producer (android-uiautomator over ADB or appium-source over Appium), a web agent-browser read, an iOS tree from a provider acquisition, or one that declares no provenance at all — because only the bridge carries the measurement. It re-prices on every capture, so a mid-run fallback to a more expensive read widens the remaining wait, and it reports the window it ran on (settle.quietMs) so the response states which default it applied.

Keying on the baseline's producer was the first cut here and is wrong: the window governs the loop's own reads, and a stored ref frame or session tree can be several commands stale or served by another producer. Keying on the channel would be wrong too — the bridge and the Apple runner share xctest and read at different costs. --settle-quiet still outranks both, and wait stable keeps the general default of its own.

Evidence for 300 ms

iPhone 17 simulator, one run per window per rep, order alternating so host load falls on both arms alike:

Screen	500 ms	300 ms	captures
text entry, Catalog search	859 ms	664 ms	3 → 2
press, product Add to cart	844 ms	647 ms	3 → 2

Accuracy held: diff snapshot -i two seconds after the settled response found no further change for the text entry at either window (0/6 both), and one further change in three of six runs for the press at either window — that screen keeps animating after the press whichever window is asked for, so the shorter window changes the wait, not the verdict's accuracy. (These A/B runs, and the live runs below, were taken on the pre-split build that still carried the presented-surface trigger since lifted to #3570; both arms request their window explicitly, so the comparison is this layer's window arithmetic alone.)

Validation

Tested at 84f170e31. settle-default-quiet.test.ts pins the window per capture producer in both mismatch directions (bridge baseline + runner re-read widens to 500 ms; runner baseline + bridge re-read narrows to 300 ms) and pins flag precedence; mutation check, measured when the key moved (6f43f54e6) and unchanged in shape here: keying the rule back on the baseline fails three of those cases plus one daemon case. generic-settle.test.ts pins the three default-window routes on an injected fake clock (a 25 ms poll, a 30 ms sleep, deterministic elapsed arithmetic — no real-time budget races).

The per-capture re-pricing briefly broke the layer beneath it: requiredQuietMs received the freshly resolved default each poll instead of the previous requirement, so the 1.5 s dismissal floor survived only while each consecutive capture kept qualifying. The comparison refuses a node-less side, so one empty mid-animation projection dropped the floor and the next empty read settled the run — 302 ms after a dismissal that owed the 1.5 s floor. The floor is now latched once a capture proves the dismissal and maxed against the current window rather than replacing it (pinned by a dismissal confirmation is not dropped by a later empty projection; fails at 302 ms against the pre-fix head, passes against layer 1's source, which held the floor only by accumulating).

Live on the same simulator after the rework: default press --settle reports quietMs: 300 at 658–664 ms over two captures; --settle-quiet 700 still wins at 1050–1076 ms; the layer-1 dismissal floor is intact (quietMs: 300 with waitedMs: 1875/1886, i.e. the 1500 ms floor on top of the narrower window); wait stable is unaffected. Two provenance notes on those runs. They predate the latch commit: they held the floor because their captures kept qualifying capture-to-capture, and the latch changes behaviour only where one stops qualifying mid-confirmation, which the unit case above covers. And they were taken on the pre-split build, which still carried the presented-surface trigger since lifted to #3570; at this head the trigger is main's, which arms on a superset of transitions, so the floor these runs show can only hold at least as well here — the window arithmetic they measure (quietMs, flag precedence, capture counts) is this layer's and unchanged.

Gate

pnpm check:affected --run --base origin/main at 84f170e31: all runnable checks passed (exit 0) — 514 related test files / 3,998 tests, Fallow clean on the 8 changed files — held under the machine-wide lock, started 2026-10-13T10:56:14Z. format, lint, tsc -b, check:layering, build and vitest related all pass; the 1500 → 1500ms unit fix in the MCP/Node field description is covered by the schema/parser/registry suites (47 files, 309 tests) with no text pin against it.

Note on scope

An earlier revision of this stack carried, one layer down, a presented-surface arm that keyed on the baseline's node count — the defect found in #3512's review (#3512 (comment)), where a dismissal whose baseline is a full tree stops confirming. That arm is not in this stack anymore: #3512 is now the window decoupling alone and keeps main's trigger, and the narrowing (with that rule and the measured route table) sits on top of this layer as the draft #3570, blocked by #3565. So this layer arms exactly where main arms, only at every requested window, and nothing here depends on how #3565 resolves the fact.

[github-actions]

Size Report

Metric	Base	Current	Diff
Installed (including dependencies)	5.18 MB	5.18 MB	-2.8 kB
Package (unpacked)	5.18 MB	5.18 MB	-2.8 kB
Package (download)	1.56 MB	1.56 MB	-365 B

Startup median (7 runs, lower is better):

Scenario	Base	Current	Diff
CLI --version	27.8 ms	28.2 ms	+0.4 ms
CLI --help	78.1 ms	77.7 ms	-0.4 ms

[cubic-dev-ai]

All reported issues were addressed across 9 files

<sub>Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic</sub>

[thymikee]

The settle change looks correct at 6ad9bf3, and CI is green, including Smoke Tests, with no conflicts. Before merge, please decide who owns the quiet-window fact and fix the 300 ms scope text so it names the simulator snapshot bridge. This PR is also stacked on #3512, which should land first.

Not blocking, and you can take or leave these: (1) the window is keyed on the baseline's producer at https://github.com/callstack/agent-device/blob/6ad9bf3/src/commands/interaction/runtime/settle.ts#L175, but the loop compares the loop's own captures. The baseline can come from an earlier stored snapshot, and the bridge can fall back to the Apple runner mid-session. A bridge baseline followed by runner captures then gets 300 ms, and a runner baseline followed by bridge captures misses the speedup. The rule should be that the default window follows the source of the captures being compared. (2) Help, grammar and client docs say 300 ms applies to an iOS simulator or healthy iOS interactive tree (https://github.com/callstack/agent-device/blob/6ad9bf3/packages/command-registry/src/flag-definitions-action.ts#L214), but the code applies it only to the simulator snapshot bridge. A simulator capture served by the Apple runner keeps 500 ms. Deriving that text from the same constant would keep it honest. (3) The roughly 17-line benchmark comment on SIMULATOR_BRIDGE_SETTLE_QUIET_MS could shrink to two lines, with the measurements moved to docs/evidence. (4) The two new scroll --settle cases in generic-settle.test.ts use the real settle clock, about 900 ms of sleeps. An injected clock would match settle-default-quiet.test.ts.

Could settleAfterAction pass quietMs as optional, so that runStableCaptureLoop resolves the default from the first capture's provenance? The 300 ms value would sit beside the producer table in ios-snapshot-acquisition.ts, or be exported from platform-apple, instead of a literal in shared runtime. That would remove SurfaceScopedNodes.captureSource, both writers and the generic-route threading, and the loop could be tested directly. One owner would cover both routes and follow the source actually read, including after runner fallback. pollMs would then be derived after the first capture, as it already is from quietMs. Nothing blocks this, though the quietMs and pollMs setup would become lazy.

The open inline threads on the 300 ms wording at #3522 (comment), #3522 (comment) and #3522 (comment), and on the baseline-keyed window at #3522 (comment), still stand. The two lower-priority threads on the benchmark comment and the real-clock tests (#3522 (comment) and #3522 (comment)) also still apply. The first two are marked resolved, but the text is unchanged at this commit.

I did not rerun the live iPhone 17 A/B or the affected-checks run, so the PR body's measurements are unverified here. I also did not exercise a mid-loop fallback to the runner. Point 1 comes from reading the fallback paths in snapshot-route.ts.

This review covers 6ad9bf3. The branch has since moved to 9bf62c7, a rebase onto the updated #3512 that only renames a test in the base. I will confirm the review against the new head on the next pass.

[cubic-dev-ai]

All reported issues were addressed across 10 files (changes from recent commits).

<sub>Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic</sub>

[thymikee]

Fixed in 6f43f54e6, which the run predates — the block ran from 17 lines of benchmark data down to a rule statement, and no device, timing, or run count remains in the source:

/**
 * The default quiet window a capture earns. The iOS simulator snapshot bridge answers well inside
 * the general default, so holding the general window on that read spends a whole extra capture on a
 * window its tree had already satisfied. Keyed on the producer, not the channel: the bridge and the
 * Apple runner share the `xctest` channel and read at different costs, and only the bridge carries
 * the measurement. Every other read — the runner's own tree read, the Android helper, a capture that
 * declares no provenance — keeps {@link DEFAULT_STABLE_QUIET_MS}.
 * (PR #3522 holds the interleaved before/after medians and the late-diff accuracy check.)
 */

I kept the numbers in this PR's description rather than opening a file under docs/evidence/: that directory's existing entries are exact-head sweeps with raw artifacts, digests, and named corpus tags, and this is a six-sample manual A/B on one simulator. Pointing at the PR number does rot once this merges, so the pointer stays a parenthetical rather than a {@link} anyone is meant to resolve.

[thymikee]

The code in 88fea6f looks right to me, but I would like the latch commit covered by one live run before merge. The CI checks are green (14 reported, none failing), and there are no conflicts.

The live simulator runs in the PR body predate the latch commit, so they do not show it working. Please run one live simulator settle that reaches a confirmed dismissal, then keeps polling on the 300 ms bridge window. The output should show the quiet window staying at 300 ms for the rest of the run. I read the code and the test fixtures by hand. I did not run the mutation checks, and I did not read every line of the fake-clock wiring in generic-settle.test.ts.

Not blocking: defaultQuietMsForCapture compares the producer string and keeps the 300 ms literal in shared runtime, while capture-kit already has exhaustive per-producer tables (ios-snapshot-acquisition.ts), and the latch comment cites fixture numbers (150 ms, 302 ms) where it should state the invariant that a proven dismissal holds for the rest of the run. You can take or leave both.

Could the quiet window be one exhaustive settleQuietMs table next to that truncation table, with 500 for undefined provenance and non-iOS producers? The loop would then only call that accessor, and most of the extra lines left would be the pacing restructure, which could be its own refactor.

Of the eight bot threads, all eight look fixed at this head, so you can resolve them: r4250825442, r4250825462, r4250825469, r4250825474, r4250825481, r4250825483, r4251997139 and r4251997145.

Before merge, #3512 (layer 1) must land first, including its decision on the presented-surface fact, and this PR must then be retargeted.

[thymikee]

Gate — fix/3470-settle-quiet-per-capture-backend @ 84f170e31

Supersedes the earlier gate comment on this PR, which was taken at b78640a35 before #3482 and #3512 merged and this layer was restacked onto main.

Local, held under the machine-wide lock, started 2026-10-13T10:56:14Z, against a clean tree.

pnpm install --frozen-lockfile              exit=0
pnpm build                                  exit=0
pnpm format                                 exit=0  (tree clean afterwards)
pnpm check:affected --run --base origin/main exit=0

check:affected selected the affected set (7 checks, no fail-open) and reported all runnable checks passed: format:check, lint, typecheck (tsc -b), check:layering, check:fallow --base origin/main (No issues in 8 changed files, 22.78 s), build, and vitest related over the 8 changed files — 514 files / 3,998 tests.

What the restack changed in this layer's own files: generic-settle.test.ts only. Its dispatchGeneric helper had lost its session parameter on main (#3510 rewrote that helper's runtime-binding seam), so this layer's three new default-window cases now seed the session and omit the argument — the helper's body never read it. The post-action-observation-grammar.ts description also picks up the reviewer's 1500 → 1500ms unit fix on the line this layer already rewrote. Nothing else in the 8 files differs from b78640a35.

[thymikee]

This follow-up on 88fea6f covers b78640a. The earlier review items are fixed in the code, but one regression and the missing live run remain.

Fixed: the quiet window now re-prices from each loop capture, the confirmation latch is present, and the doc and test comments are corrected. The earlier review asked for a live latch run, and none was posted for this head.

The remaining problem is in broadTransitionConfirmation. It decides arming once and stores null after the first capture. But broadTransitionBaseline reads per-capture inputs, capture.nodes.some(isViewportRootNode) and capture.backend. The base re-ran this on every capture until one armed. Take a dismissal whose pre-action tree has more than 5 nodes, such as a full-screen modal or form sheet. If the first post-action capture is rootless or empty mid-animation, the head rules the confirmation out for the whole run. The base would arm on the next rooted capture. So an iOS dismissal that the base and main hold for the 1.5 s floor now settles on the 300 ms window, which is the #1831 transitional-tree case the floor exists to prevent. It also means this layer does not arm exactly where main arms, as the PR body says. The rule: a run may keep the arming answer only from inputs that are constant for the run, which are the baseline nodes, the platform and the app bundle id. Store null only for those, and re-check the per-capture completeness and backend on each capture until one arms. Please add a regression with a baseline above 5 nodes, a rootless or empty first capture, then rooted replacement captures, and require it to hold for at least 1500 ms. I read this from the code and did not run it against either commit, and I have not seen on a device that a non-tiny modal dismissal gives a rootless first capture. Does the base comment about a post-dismissal capture omitting the viewport root match what you see?

After that fix, please post one live iPhone simulator run of press --settle with the default window (no --settle-quiet), dismissing a presented modal into a full screen. The response must show settle.quietMs 300 and waitedMs of at least 1500. Include the response JSON and the head sha. The PR body says the current live runs predate the latch commit and use the earlier trigger.

On the open inline threads, the latch on confirmedQuietMs is fixed at this head, but it depends on the arming change above. These threads are fixed at this head and can be resolved: simulator bridge naming, 300ms scope in client-gesture, re-pricing per capture, flag help naming the bridge, removed benchmark block, fakeClock in the settle test, and quietMs doc on every return.

Smoke Tests is still running and no failure has been reported. This change touches the settle loop that the --settle smoke steps exercise, so a later failure there could relate to it. Before merge, the arming check needs to re-run on each capture until one arms, the rootless-first-capture regression needs to be added, and the live default-window modal-dismissal run needs to be posted. I did not run the mutation checks you describe.

[thymikee]

Smoke Tests on this layer — investigation, per the note on #3512

The standing guidance is that a smoke failure on a settle diff is probably related, so this records what each failure actually was rather than only that it went green. The iOS lane is the only lane that has failed on this layer's history; Android, macOS and Linux are green at every head that has reported one.

Head 93e79ccb2 (08:01Z) — job/115192846055, attempt 1, still red, never rerun. Failing step Run fixture-backed iOS simulator E2E smoke, inside assertRegularVisibleDepthFrontier (test/integration/ios-simulator-e2e/live-snapshot-depth-frontier.ts:19), which is the scenario's first command:

agent-device open com.callstack.agentdevicelab --relaunch
  --launch-url agent-device-test-app:///snapshot-depth --udid 5FEB61C5-…
→ code COMMAND_FAILED, "Simulator device failed to open agent-device-test-app:///snapshot-depth."
  NSPOSIXErrorDomain code=60 — Operation timed out (ETIMEDOUT from xcrun simctl openurl)

This one is dispatched, so I will not wave it away: the timeout is simctl handing a URL to a freshly relaunched simulator, before any capture is taken and therefore before runStableCaptureLoop, the --settle default, or the window resolution this PR changes can be reached. That is a reachability argument, not a passing rerun — this specific job has no second attempt.

Head b78640a35 (09:06Z) — run 38373920347, iOS lane, now showing green on attempt 3. Two red attempts sit behind that badge (attempt 1: IOS_RUNNER_CONNECT_TIMEOUT + runnerConnectFailureReason: xcodebuild_exited_early, dispatched: "no", xcodebuild's Test crashed with signal kill before establishing connection; attempt 2: kind: daemon_startup_failed, startupTimeoutMs: 15000), both dying in Preflight iOS runner through public CLI (prepare ios-runner) before any command is dispatched. Attempt 3 went green at 10:21Z.

Same tree, three failures and one pass. 93e79ccb2 and b78640a35 share a tree (645cdcdd2…, git diff between them is empty), so all four iOS attempts above ran identical content: it failed once on a dispatched simctl openurl timeout, then twice pre-dispatch on preflight, then passed. The honest summary is that this lane is unreliable for this content, not that any single failure has been root-caused to the host.

On the reruns themselves: I reran, and that was the wrong order for the 93e79ccb2 failure. The two preflight signatures justified a rerun on reachability grounds; the deep-link timeout did not, and its lane was never retried. Recording it here so the green badge at b78640a35 is not read as covering 93e79ccb2, and so the count of attempts on this lane is visible: this layer has consumed three iOS attempts to get one pass.

Still owed: the iOS lane re-running at 84f170e31 (content delta from b78640a35: test-helper argument adaptation plus the 1500 → 1500ms help-text fix) was queued when this was written, along with everything else on the pool. I'll append the result here. If a smoke failure comes back inside a settle or interaction step, or with dispatched: "yes", I stop and treat it as diff-caused.

[thymikee]

The 300 ms default window is not safe yet, because one of the two problems from the earlier review at b78640a is still open at 84f170e. The thread fixes and the latch are in. Checks are still pending, and no conflicts are known.

The arming decision is still made once, on the first capture. In stable-capture.ts, broadTransitionConfirmation is set on the first capture and stored as null for the rest of the run. broadTransitionBaseline returns null on per-capture inputs: capture.nodes.some(isViewportRootNode) and capture.backend. Take an iOS dismissal whose pre-action tree has more than 5 nodes, such as a full-screen modal or form sheet. If its first post-action capture is rootless or empty mid-animation, or reports a non-XCTest backend, the confirmation is ruled out for good. A default press --settle then settles on the 300 ms window instead of the 1.5 s floor that base and main hold. That is the #1831 transitional-tree case the floor exists for, so the PR body's claim that it arms where main arms does not hold. The rule: the run may keep the arming answer only from inputs that stay constant for the run (baseline nodes, bound platform, appBundleId), and may rule out with null only on those. Viewport-root completeness and the capture backend must be re-checked on every capture until one arms. Please add a regression with a baseline above 5 nodes, a rootless or empty first capture, then rooted replacement captures, and assert waitedMs >= 1500. It should fail at 84f170e.

Could the arming question be split by input lifetime? Rule out once from the run-constant inputs, then check completeness and backend on each capture inside requiredQuietMs until one capture arms and latches. That is the base's ??= re-check behavior plus the new latch, and it removes the need for the null tri-state comment.

This change touches the device-facing iOS settle path, and the latch with per-capture re-pricing has not been run on a device. The live runs in the PR body predate the latch commit and used the earlier trigger. Once the fix above is in, please post one iPhone simulator press --settle with no --settle-quiet, dismissing a presented modal into a full screen, on the new head sha. Include the response JSON showing settle.quietMs 300 and waitedMs >= 1500, plus the capture producer (simulator-ax-bridge).

Eight earlier inline threads from the other review are fixed at this commit and can be resolved: grammar description, client-gesture scope, per-capture price, flag description, doc block, injected clock, confirmedQuietMs latch and quietMs doc. The latch thread is fixed in form, but it only helps once arming succeeds, which is the first finding above.

I read the base and head code paths and did not run the tests or reproduce the first finding. I have not seen on a device that a non-tiny modal dismissal gives a rootless or empty first capture; the base code's comment is the only source for that. I did not check the mutation results or the 302 ms pre-fix measurement. All 14 non-passing checks were still queued when I looked, and none had failed. The diff changes runStableCaptureLoop, which the --settle steps in the iOS Smoke Tests lane exercise, so a later failure inside a settle or interaction step would overlap this diff. The earlier red runs (simctl openurl ETIMEDOUT, runner preflight) happened before any capture, per your notes. There are no conflicts.

Before merge, please re-check per-capture arming on every capture, add the regression for a non-tiny baseline with a rootless first capture, and post the live default-window modal-dismissal run on that head.