Skip to content

feat(ios): publish the presented surface on every iOS capture (#3565) - #3579

Open
thymikee wants to merge 1 commit into
feat/3565-presented-surface-rulefrom
feat/3565-presented-surface-publication
Open

thymikee wants to merge 1 commit into
feat/3565-presented-surface-rulefrom
feat/3565-presented-surface-publication

Conversation

@thymikee

@thymikee thymikee commented Oct 13, 2026 •

Copy link
Copy Markdown
Member

Summary

Makes the Apple runner answer the presented-surface fact in Swift, the same way layer 2's TypeScript rule
answers it (#3565). Layer 3 of 3 and the layer that closes the issue: from here every iOS capture route
carries presentedSurface — the daemon-side engines from layer 2, and the runner's own tiers from here —
so #1831's dismissal confirmation can arm on a producer fact instead of tree shape. 6 files, +515/−13.
Consumer-side narrowing stays in #3470.

On the runner, the classification happens inside makeSnapshotBackendCapture, which now takes the
acquisition beside the presentation it produced. That seam is where a tier's own tree is in hand and
in-bundle unit tests reach it, so the fact cannot go missing by reading the presented tree or by deleting
one line there. It answers unknown when the tier's nodes array is missing or empty, and
presented/unknown when nodes carry no class names, which is every runner tier today. The fact crosses
the wire as text because a Decodable Bool? decodes "presented" into true, which the shared
reader refuses; the Swift test decodes through the real JSONValue + wire reader, so a round-trip
failure fails the build rather than a manual run.

The Swift rule mirrors layer 2's TypeScript rule exactly, including where evidence is matched on the node
occurrence a walk stood on rather than on the number it carries, so a keyboard host's subtree cannot
swallow an app window's own dimming view, while absent still requires numbering that identifies each
node once.

The Swift rule and its parity test are here rather than with the TypeScript rule that shares their golden
table, for two reasons that are the repository's gates rather than preference: layer 2 needed the
publication to give its rule a production consumer (check:production-exports), and carrying the Swift
half as well would have put it at 1,242 gross lines against the 1,000-line budget. The table itself is in
layer 2, so this parity test is checked against a fixture that already exists — the Swift half cannot
silently redefine a case.

Evidence coverage (iPhone 17 simulator, RN test app, head c06d8ec9a)

Rows marked live were read off the device at this head. The rest are golden cases, said so plainly.

Presentation Answer Basis
Home, Catalog, automation lab — one content window, one container absent live, --raw and every projection
The same lab after the sheet is dismissed absent live
RN pageSheet sheet — window holds 2 containers, dimming views at 4 and 134 presented live
That sheet through the projected cut (6 nodes) and the interactive cut (4 nodes), neither of which publishes a dimming view presented live — the fact is read off the acquisition, not the projection
Home with the Expo dev-menu floating button on screen (a second content window) unknown live — a window this rule cannot name
Any screen through the private-AX backend (--actions, 0 class names published) unknown live — the class-blind gap
Sheet resting at an undimmed detent presented golden case, not device-captured
RN Alert.alert (own window, no dimming view) presented golden case, read from a captured tree
Alert whose producer names only Element(7) presented golden case, both readings agree
Full-screen / custom modal, no dimming view unknown golden case — nothing positive, no guessing
SwiftUI .sheet presented expected same UIKit container as a page sheet; not device-captured
SpringBoard system alert out of scope resolveBlockingSystemModal stamps .presented (layer 1)
Any XCTest runner tier (no class names published) unknown, or presented on an element type alone in-bundle runner tests, not device-captured
Capped, unstated, malformed numbering unknown, or presented on evidence it did read golden cases

Two things the live run is worth stating out loud. The dev-menu floating button puts a real
EXDevMenu.DevMenuFABWindow beside the app window, and the honest answer for that screen is unknown —
the four absent rows above were taken with it disabled
(defaults write com.callstack.agentdevicelab EXDevMenuShowFloatingActionButton -bool NO), which is the
single-content-window shape the rule reads absence from. And the interactive cut of the open sheet
publishes four nodes, none of them a dimming view, while the fact still answers presented: that is the
failure mode this fact replaces, since a consumer reading the published tree would have had to answer
absent on a screen with a sheet over it.

Validation

c06d8ec9a, verified at this head: swift test runs the parity suite at 23/23 against the layer-2
fixture, check:production-exports passes, pnpm build and pnpm typecheck are clean, and vitest over
packages/capture-kit, packages/kernel and packages/platform-apple is 2,963 green, eager-closure
budgets included. The runner's own suite runs on the iPhone 17 simulator: 430 tests, with testCommand
the one failure — it refuses to listen without the listener port and token the daemon injects, so it
cannot pass under a bare xcodebuild test run, and this stack does not go near that file. The live
capture table above is from the same head.

The repository-wide local gate for this head is deliberately not claimed here: four other worktrees on
this host run subprocess-heavy suites concurrently, and the earlier local failures in this stack — an ADB
connect failure, a daemon startup timeout, one abort-timing case — were contention from exactly that,
each passing in isolation (47/47, 6/6) while failing under load. GitHub CI is still queued at this head;
the Compatibility & Provenance lane, which the previous head genuinely failed, is the one worth reading
on this PR first — its check is reproduced locally above.

Both parity tests read contracts/fixtures/ios-presented-surface.json, the single cross-language table
from layer 2, so either language answering one of these 18 cases differently fails its own suite, and the
cross-language claim is enforced rather than asserted. The runner seam is pinned in
RunnerTests+SnapshotCapturePlanTests.swift: a missing nodes array answers unknown with
presentedSurfacePresented nil, a bare Sheet element type answers presented (a positive is strong
even with no class names), and an unnamed tree answers presented/unknown and never absent. That
seam's test now also asserts its own premise — that the presentation the tier produced does not carry the
node the evidence sits on — so the assertion below it cannot silently become a test that reads one tree
twice; swapping the evidence node for one the projection retains fails that premise assertion.

@thymikee
thymikee added this pull request to stack #3580 October 13, 2026 12:50

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 6 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread packages/capture-kit/src/ios-snapshot-engine/engine.test.ts Outdated
@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 5.20 MB 5.21 MB +14.2 kB
Package (unpacked) 5.20 MB 5.21 MB +14.2 kB
Package (download) 1.56 MB 1.57 MB +4.3 kB

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 28.7 ms 29.3 ms +0.6 ms
CLI --help 81.0 ms 81.0 ms +0.0 ms

@thymikee
thymikee force-pushed the feat/3565-presented-surface-publication branch from 719fab6 to dbae81f Compare October 13, 2026 14:13
@github-actions

github-actions Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1

QR code for preview link

🚀 View preview at
https://callstack.github.io/agent-device/pr-preview/pr-3579/

Built to branch gh-pages at 2026-10-13 19:11 UTC.
Preview will be ready when the GitHub Pages deployment is complete.

@thymikee
thymikee force-pushed the feat/3565-presented-surface-publication branch from dbae81f to 27035e5 Compare October 13, 2026 14:29
@thymikee
thymikee force-pushed the feat/3565-presented-surface-publication branch from 27035e5 to a083036 Compare October 13, 2026 14:34
@thymikee
thymikee force-pushed the feat/3565-presented-surface-publication branch from a083036 to 28076fc Compare October 13, 2026 14:46
@thymikee
thymikee force-pushed the feat/3565-presented-surface-publication branch from 28076fc to 14f5c0f Compare October 13, 2026 14:47
@thymikee
thymikee force-pushed the feat/3565-presented-surface-publication branch from 14f5c0f to 1e3f5eb Compare October 13, 2026 15:09
@thymikee
thymikee force-pushed the feat/3565-presented-surface-publication branch from 1e3f5eb to 16cf635 Compare October 13, 2026 15:28
@thymikee
thymikee force-pushed the feat/3565-presented-surface-publication branch from 16cf635 to 54a46e7 Compare October 13, 2026 15:54
@thymikee

Copy link
Copy Markdown
Member Author

This layer now carries the Swift half of the rule (SnapshotPresentedSurface.swift and its parity test) in addition to the runner stamp and the docs, and no longer carries the engine/contracts publication, which moved to #3578 so that layer's rule has a production consumer. check:production-exports and the 1,000-line budget are what forced the move; the golden table stays in #3578, so the Swift parity test here is checked against a fixture that already exists. Head is 54a46e734: 23/23 swift test, production-exports, build, typecheck, 842 eager-closure and the capture/runner suites verified at this head, 490 gross lines. The stack's final tree is byte-identical to before the re-cut.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 2 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

@thymikee

Copy link
Copy Markdown
Member Author

The runner change looks reasonable, but I cannot call it ready yet. I reviewed 54a46e7. The one gap is live evidence for the runner route.

This PR changes a device-facing path: every runner tier now publishes presentedSurface (RunnerTests+SnapshotCapturePlan.swift#L111). The "live" row for the runner tiers in the PR body was captured at 719fab6, not at this head, and it quotes no output. Nothing shows a runner-served capture carrying the field through readPresentedSurfaceFact into snapshot --json and the stored baseline. The simulator snapshot bridge is a different route, so it does not prove this one. Please run snapshot --actions --json at 54a46e7 on the iPhone 17 sim with the RN test app, once with the Open automation sheet up and once on Home. The output must show snapshotQuality.backend equal to private-ax (or a runner tier on a physical device). With the sheet up, presentedSurface must read presented or unknown. On Home it must read unknown, never absent. Please also show Alert.alert on the same route, and quote the JSON lines.

Not blocking, and you can take or leave these: the docs (snapshots.md#L234) and the Swift comment at SnapshotPresentedSurface.swift lines 44-45 say --raw, -i, -d and --scope give the same value for one screen, but on runner captures --raw -d N bounds the acquisition, so a deeper Element(7) is never seen and the same screen can read unknown instead of presented. Either scope that claim, or classify on an acquisition that ignores the raw depth, with a seam test where Element(7) sits below it. Also, the only production entry, classify(_ acquisition:) (SnapshotPresentedSurface.swift#L129), always passes a nil class name, so the window walk, system-window exclusion, alert and dimming checks and the .unstated case run only under the parity test.

Could the seam change stay as it is while SnapshotPresentedSurface.swift shrinks to the class-blind rule? That rule returns presented when any node type is a presenting type (named or Element(5..8)), and unknown otherwise. The parity test would pin presentingElementTypes and the classBlindExpected column from the fixture. That is about 30 production lines instead of 219 with the same production behavior, and the class-aware half would land with the change that first gives runner tiers class names.

Of the open review threads, two do not apply here: the engine.test.ts one (r4258846755) belongs on #3578 because that file is not in this diff, and the one on the unreachable system-window exclusion (r4260955100) cannot happen on the Swift route because class names are nil there. The third (r4258846775) is also not a defect, because the test does fail on a presented tree; only its "dropped every node" comment overstates, so please resolve it.

CI is still pending: Smoke Tests and Bundle Size show cancelled runs, which look like superseded runs, and one Smoke Tests job is still running. The smoke result on this head matters, since this change touches the payload every runner-served snapshot goes through. I could not diff 719fab6 against this head, so the "byte-identical merged stack" claim is unchecked. The live runner capture above must land before merge.

@thymikee
thymikee force-pushed the feat/3565-presented-surface-publication branch from 54a46e7 to c06d8ec Compare October 13, 2026 17:20
@thymikee

Copy link
Copy Markdown
Member Author

Review round fixed at c06d8ec9a (rule change also in layer 2 777d9b73a, where those two files now live): positive evidence is matched on the node occurrence the walk stood on rather than the number it carries, so a keyboard host's subtree no longer swallows an app window's own dimming view; absent still requires each node numbered once. The golden table gains an 18th case for the shape that genuinely is ambiguous, and the two tests called out in review now assert their own premises.

@thymikee

Copy link
Copy Markdown
Member Author

Re-ran the live device table at c06d8ec9a (iPhone 17 simulator, RN lab app) and replaced the stale 719fab62d capture in both bodies. Three rows are worth a reviewer's attention:

  • The sheet is presented through the projected (6 nodes) and interactive (4 nodes) cuts, neither of which publishes a UIDimmingView. That is the failure mode this fact replaces: a consumer reading the published tree would have answered absent on a screen with a sheet over it.
  • Home with the Expo dev-menu floating button on screen answers unknown, because EXDevMenu.DevMenuFABWindow is a real second content window. The four absent rows were taken with it disabled (defaults write com.callstack.agentdevicelab EXDevMenuShowFloatingActionButton -bool NO). The rule is right to refuse there; a reviewer reproducing the absent rows needs that default set.
  • The private-AX backend (--actions) publishes zero UIKit class names and answers unknown on every screen, which is the class-blind gap the body calls out — fix: derive iOS transition snapshots from visible presentation #1831 cannot arm a confirmation from those captures.

Add the rule that answers whether an iOS capture read a modal presentation over
the app's own content, the golden table that pins it, and the publication that
carries its answer, so the rule has a production consumer the moment it lands.

The rule reads a dimming view without its enablement, unlike the occlusion cut
that shares the same container class: a sheet resting at an undimmed detent keeps
a dimming view that takes no touches and is still presented. `absent` is the
strong answer and never the default, because a consumer arms a dismissal
confirmation on it. It requires that the producer stated the capture was not
capped, that every node carries the UIKit class name the rule reads, that each
node is numbered once so the parent links the walk follows name the node they
mean, and that the one content window under the app root holds exactly one
presentation container. Positive evidence — the dimming view a presentation owns,
an alert's own root class, or an element type that names a presentation — answers
`presented` from a single node, so a capped capture still reports what it read.
Producers that publish no class names, which is every XCTest runner tier, answer
`presented` or `unknown` and never `absent`.

The rule lives in `geometry-policy.ts` beside the occlusion cut rather than in
`engine.ts`, because that module already owns the UIKit presentation vocabulary
both readers consume and `geometry.ts` already imports it on the engine's eager
path, so the placement costs nothing and leaves one module to change when UIKit
does. Both readers now match class names through the same normalization, so a
producer that spells a class with a module prefix cannot reach one answer in the
occlusion cut and another here.

Evidence is matched on the node occurrence a walk stood on, not on the number it
carries, so a keyboard host's subtree cannot swallow an app window's own dimming
view; only `absent`, a claim about every window the producer read, requires
numbering that identifies each node once.

The publication is in this layer rather than one up because
`check:production-exports` refuses an exported rule with no production consumer,
and a layer that ships a decision nobody can read is not a layer that passes. The
Apple runner's Swift twin and its own consumer land in the next layer, where the
runner acquires trees; both languages read the one fixture published here.
@thymikee
thymikee force-pushed the feat/3565-presented-surface-publication branch from c06d8ec to 81b9bf5 Compare October 13, 2026 18:23
@thymikee

Copy link
Copy Markdown
Member Author

The code now looks right at 81b9bf5, but the live evidence is still missing, so this is not ready to merge yet. The earlier thread on per-occurrence evidence is fixed in the classifier (golden case 17 now expects presented), and the seam test now checks its own premise before it checks the answer.

The remaining gap is the runner route in RunnerTests+SnapshotCapturePlan.swift. Every runner tier now stamps presentedSurface through makeSnapshotBackendCapture, so this is the device-facing payload. The re-run reports the private-AX result as prose ("unknown on every screen") with no quoted JSON and no snapshotQuality.backend. It also skips Alert.alert on this route. Without that, nothing shows that an Alert-typed AX node reaches the classifier, or that the field survives readPresentedSurfaceFact into snapshot --json. Please run snapshot --actions --json on the iPhone 17 sim with the RN lab app, at 81b9bf5 or c06d8ec (runner behavior is the same at both). Do it on Home, on the open sheet, and on Alert.alert. Quote the JSON lines that show snapshotQuality.backend == "private-ax" and presentedSurface for each. Home should read unknown, never absent. The sheet should read presented or unknown. The alert should read presented. If the alert reads unknown, please say which element type the AX tree published for it.

Checks are still pending. The three non-passing checks were cancelled as superseded runs, not failed, so they give no signal at this head. The iOS Smoke Tests job does overlap this diff, because it runs the runner snapshot payload that this PR extends. Bundle Size does not overlap, since the diff is Swift and docs only. I did not run swift test on apple/snapshot-presentation at this head. The 23/23 parity claim predates the 19th fixture case and the Swift normalization, so please re-run it.

On the other threads, the AX-node thread on the Alert classifier is fixed at this head, so you can resolve it. The engine.test.ts thread does not apply here, because that file belongs to base #3578 and is not in this diff. The seam test thread is fixed by the premise assertion, so you can resolve it too.

Small notes, not required for merge. The head commit message carries the layer 2 subject instead of this PR's. The doc comment at SnapshotPresentedSurface.swift:47-48 still points to collectPresentedSurface in engine.ts, though it now lives in geometry-policy.ts. The PR body still says 18 cases.

The next step before merge is the quoted runner-route JSON for Home, the sheet and Alert.alert.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant