Skip to content

feat(snapshot)!: make a node's ref optional and mint it per producer - #3547

Merged
thymikee merged 6 commits into
mainfrom
feat/snapshot-optional-refs
Oct 13, 2026
Merged

thymikee merged 6 commits into
mainfrom
feat/snapshot-optional-refs

Conversation

@thymikee

@thymikee thymikee commented Oct 12, 2026 •

Copy link
Copy Markdown
Member

Summary

SnapshotNode.ref was declared always present, which was already a lie the moment a capture's producer resolved actions against refs of its own: a node the producer issued no ref for has one missing, and every surface that prints a ref printed @undefined for it.

ref becomes optional and attachRefs takes the capture's minting mode:

snapshotRefMinting({ backend: 'web', producer: 'agent-browser' })     // 'backend-issued' — keeps the producer's refs, mints none
snapshotRefMinting({ backend: 'android', producer: 'appium-source' }) // 'dense' — numbers the tree exactly as before

snapshotRefMinting keys on the capture's producer, not its platform channel, and takes the whole provenance pair rather than a partial one. A channel is not a producer — the provenance table's own rule is that several producers can feed one channel with different guarantees, and only agent-browser resolves actions against refs of its own. A second adapter that lands on the web channel mints no refs, and a channel-keyed check would have moved it to backend-issued and silently stripped the refs its whole tree depends on (see the review-round comment below).

hasSnapshotRef and snapshotRefBodies are the one way to read the absence, and every surface that offers a node as an address now reads it: overlay labels, the printed snapshot line, settle tails, MCP ref pins, the touch-point competitor disclosure, keyboard occlusion, find/get reads, and the record- and replay-time target classifiers, whose ref evidence a ref-less node cannot contribute. No producer emits a ref-less node in this PR, so behavior is unchanged for every backend; the contract and its consumers land first so the capture that needs it (#3515) lands on top. 27 files, 582 gross lines.

Breaking for TypeScript consumers, source-level. SnapshotNode.ref is in the shipped declarations (dist/src/sdk-contracts.d.ts, re-exported as type SnapshotNode from dist/src/index.d.ts, and reached through SnapshotState.nodes / CaptureSnapshotResult.nodes / agent-device/contracts). Code that reads it as always present — const r: string = node.ref, a SnapshotNode passed where ref is required, node.ref.length — typechecks today and fails here; examples/sdk/client-session.ts:68 is that exact case, now guarded with target?.ref !== undefined before press({ ref: target.ref }). The repo has no changeset and no CHANGELOG (release.yml:203 generates notes from the commits since the last tag), so the record is the ! in the subject — feat(snapshot)!: … — the way dbf113a55 refactor(platform-android)!: … (#3023) carried one. Runtime behavior for every existing backend is unchanged; the version bump stays with the release owner.

Review round 2: the minting mode keys on the producer, not the channel

A cold review asked what snapshotRefMinting's provenance?.backend === 'web' does to a non-agent-browser web adapter (a webdriver- or Playwright-backed one) that publishes backend: 'web': it would flip to backend-issued and every node it publishes would lose the ref its actions resolve against. The axis was wrong and is fixed; the silent-stripping half has no live victim. Full evidence in the review-round comment.

  • Wrong axis, now at the owning type. This file's own provenance table says one channel is fed by several producers with different guarantees and that presentation/geometry logic must key on the producer, never the channel — xctest alone has four. REF_NAMESPACE_OWNERS now names the producers that own their ref namespace, and the parameter is the whole pair (SnapshotCaptureProvenance) instead of Partial<SnapshotProvenance>, so a caller cannot hand over a channel and leave this to guess who acquired the tree. That is pinned with @ts-expect-error, which fails pnpm typecheck if the parameter widens back.
  • No second web producer exists. src/core/interactors/web.ts is the only publisher of backend: 'web' and it stamps producer: 'agent-browser'. provider-webdriver is typed Extract<SnapshotResult['backend'], 'android' | 'xctest'>, provider-limrun publishes 'xctest'/'limrun-ios-tree', provider-testmu has no snapshot producer, and there is no Playwright/Puppeteer/webdriverio dependency in the workspace. { backend: 'web', producer: <other> } is already a compile error (snapshot-provenance.test.ts).
  • Behavior for every existing producer is unchanged, and is main's behavior: dense numbering. What changes is that the next adapter on the web channel gets dense numbering by default instead of having its refs stripped.
  • Tests: one case per entry in the provenance table (all nine non-agent-browser pairs assert dense), typed satisfies readonly SnapshotProvenance[] so a producer added to the table arrives with a case, plus a case pinning that a second web producer stays dense. Mutation-checked: reverting to the channel check fails exactly that one test.

Validation

Tested at e67b761af, rebased on origin/main (stack: #3515 → #3533 restacked on this head). pnpm check:affected --run --base origin/main on that commit under the machine-wide gate lock: all runnable checks passed; pnpm vitest run over kernel/selectors/capture-kit/replay-port/interaction/mcp/platform-web green; provider-integration green. Kernel tests cover both minting modes, the per-producer decision, and the addressability helpers. 28 files, 586 gross lines.

Unchanged-behavior proof: git diff --name-only origin/main...HEAD | grep -c '^packages/platform-' → 0, so no backend's capture path changes; the two test/integration/provider-scenarios/ files are the emulated page the contract is asserted against. "Every address surface reads the absence" is a completeness claim, so it carries its grep. grep -rn '@${.*.ref}' src packages --exclude-dir=dist-types leaves three families that print without a guard, and each is closed by a fact rather than by luck: the **acting** sites (find.ts:200and:426, target-visibility-stages.ts:45, resolution-disclosure.ts:87) take a node from a row declaring requireRect: true, and a ref-less node carries no rect— the invariant now stated on thereffield, and proven end to end by #3515's router case whereclick text=refuses withselector_not_found; the **ref-lookup** sites (ref-target-resolution.ts:124and:130, selector-read.ts:172) receive the ref as input; the two **collapsed-leaf warnings** (capture/runtime/snapshot.ts:381, quality-warnings.ts:101) require a label, and the web projection publishes ref-less text on value. The remaining prints (snapshot-lines.ts:66, settle-output.ts:111) are guarded, and the settle tail filters on hasSnapshotRef`.

That acting-site safety is a producer fact stated in the ref JSDoc, not yet a type fact: the pipeline rows declaring requireRect: true still carry a plain SnapshotNode. Typing them as SnapshotNode & { ref: string } needs a narrowing entry at the pipeline boundary rather than at the resolver, and I measured that separately rather than bundling it here — tracked as #3576, which records the attempt's error census (28 across 5 files) and why the guards relocate inward.

Risk: this widens a type every backend reads, so a consumer merged into main before this lands could assume a non-optional ref; typecheck catches it. The rebase onto main after #3482 landed is the evidence: its own new keyboard-chrome assertion (packages/capture-kit/src/snapshot-chrome.test.ts) read node.ref as required and went red at TS2345, fixed here by the file's existing guard idiom — assert the ref is there, then use it.

View guided diff Turn on auto-fix

@github-actions

github-actions Bot commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Size Report

Metric Base Current Diff
Installed (including dependencies) 5.18 MB 5.19 MB +1.9 kB
Package (unpacked) 5.18 MB 5.19 MB +1.9 kB
Package (download) 1.56 MB 1.56 MB +721 B

Startup median (7 runs, lower is better):

Scenario Base Current Diff
CLI --version 26.9 ms 26.1 ms -0.9 ms
CLI --help 77.6 ms 79.2 ms +1.6 ms

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 24 files

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread src/commands/interaction/runtime/selector-read.ts Outdated
Comment thread packages/kernel/src/snapshot.ts
@thymikee

Copy link
Copy Markdown
Member Author

Why this exists as its own PR

The coordinator flagged #3515 at 1,031 gross lines (git diff --stat origin/main...HEAD), over the repo's 1,000-line budget in docs/agents/pull-requests.md. #3515 is now two branches, split at the seam rather than through a hunk: the @undefined/addressless-node presentation work cannot stand apart from making ref optional — that optionality is exactly what stops those sites typechecking — so it travels with the contract it belongs to, and the web capture that first produces such a node lands on top of it.

branch PR gross
feat/snapshot-optional-refs #3547 388 (24 files)
fix/web-snapshot-refless-text (this PR's head's base) #3515 656 (5 files)

Content is unchanged by the split: against the same merge-base, the old head a1d1eb862 and the two branches together touch an identical 29-file set, with identical per-file insert/delete counts on 27 of them (git diff --numstat on each side, sorted and diffed). The two exceptions are packages/kernel/src/snapshot.ts and src/commands/interaction/runtime/selector-read.ts, which carry this round's JSDoc edits, and the only structural difference is that the 5 web-side files now live on #3515 while the 24 contract/presentation files live here.

0c4af434c (docs(snapshot): say what each ref-minting mode means where a ref is read) closes the two documentation findings from #3547's first review round, so it sits on this branch rather than as a third layer.

@thymikee

Copy link
Copy Markdown
Member Author

Review round 2: the minting mode keyed on the channel, not the producer

A cold review flagged that snapshotRefMinting asked provenance?.backend === 'web', and asked what that does to a non-agent-browser web adapter (webdriverio / Playwright / a Maestro web adapter) that publishes backend: 'web': it would flip to backend-issued and every node it publishes would lose the ref its actions resolve against.

Verdict: the axis was wrong, and it is fixed (ad579e01). The silent-stripping half is not reachable today — there is no second web producer to strip.

What the check actually looked at

The provenance table in this very file states the rule the check broke: "One channel is fed by several producers with different guarantees… Logic that assumes presentation, scope, or geometry guarantees must key on the producer, never on the channel alone." xctest alone carries four producers on one channel. Keying ref ownership on the channel borrowed an axis the table says not to borrow, and Partial<SnapshotProvenance> is what made that expressible: a partial pair lets a caller hand over a channel and leave the function to infer the rest.

The mode is now a property of the producer (REF_NAMESPACE_OWNERS), and the parameter is the whole pair or nothing (SnapshotCaptureProvenance), so the channel-alone call no longer typechecks. That negative is pinned with @ts-expect-error, which fails pnpm typecheck if the parameter ever widens back.

The three things the review asked to verify

1. Which adapters publish backend: 'web', and with what producer?
Exactly one, and it stamps the pair itself: src/core/interactors/web.ts returns { backend: 'web', producer: 'agent-browser' }. grep -rn "backend: 'web'" --include="*.ts" src packages | grep -v test is that interactor, the kernel table, and the web arms of channel-keyed switches that mint no refs.

The other providers were the plausible candidates and none of them is on this channel:

provider publishes
provider-webdriver (Appium / webdriverio) backend: 'android' or 'xctest', producer: 'appium-source' — webdriver-interactor.ts types its backend Extract<SnapshotResult['backend'], 'android' | 'xctest'>, so 'web' does not compile
provider-limrun backend: 'xctest', producer: 'limrun-ios-tree'
provider-testmu no snapshot producer at all (connection verification only)
platform-web the managed agent-browser install — it is the producer, not an alternative to it

And the pair is not writable by a foreign adapter: { backend: 'web', producer: 'anything-else' } is a compile error, already pinned by snapshot-provenance.test.ts (@ts-expect-error the web channel has exactly one producer). No Playwright, Puppeteer or webdriverio dependency exists anywhere in the workspace (grep -rn "playwright\|puppeteer\|webdriverio" package.json packages/*/package.json → 0). Maestro is a replay layer over daemon snapshots: MaestroPlatform does list 'web' for target selection, but the one place it re-attaches refs (daemon-runtime-port-observation.ts) is declared Extract<MaestroPlatform, 'ios' | 'android'>.

2. Are they reachable from the product surface?
No. The web seam is WebProvider, and resolveWebProvider falls back to the managed agent-browser provider; docs/agents/web-backend.md states web commands resolve the backend only from the state-dir managed install. An injected WebProvider still has its snapshot stamped producer: 'agent-browser' by the interactor, because that field describes the seam's contract (a ref-resolving backend) rather than the object plugged into it. The reachable set today is {agent-browser}.

3. What does main do for them?
Dense numbering, unconditionally: ref: node.ref ?? \e${idx + 1}`, with refnon-optional. So for every producer that exists,denseis main's behavior anddenseis what the fix now gives them — including any producer a later PR adds to theweb` channel, which is the case that motivated the change. No existing capture's minting mode moves; the fix removes a trap rather than altering a backend.

The tests

snapshotRefMinting now has one case per entry in the provenance table — all nine non-agent-browser channel/producer pairs assert dense, agent-browser asserts backend-issued, an absent pair asserts dense — plus the @ts-expect-error pin that a channel cannot be presented alone. The table cases are typed satisfies readonly SnapshotProvenance[], so a producer added to the table arrives with a case rather than by omission.

Those table cases cannot by themselves catch the old code, because every pair the table admits today agrees between the two axes. The case that discriminates is a second producer on the web channel keeps dense numbering, which widens the pair through one as unknown as SnapshotProvenance and asserts it stays dense. Mutation-checked: reverting the body to provenance?.backend === 'web' fails exactly that one test and leaves the other 17 green.

Why fix a defect with no live victim

Because the correction is the type, not a guard: a wrong axis plus a partial parameter is precisely the shape that makes the next web adapter's refs disappear, and the fix is that parameter's type plus one lookup list. It also means whoever adds a web adapter never has to know to look here — REF_NAMESPACE_OWNERS names itself as the site, and the default for a producer that mints nothing is the safe one.

@thymikee

Copy link
Copy Markdown
Member Author

I reviewed ad579e0. The change looks right, but I would like one more test and an answer to a design question before merge.

I ran no tests myself. I read the code, and I judged the mutation claim (the channel check fails exactly one test) from that reading. The rect-implies-ref rule relies on the producer in #3515, which is not in this diff. I also did not check whether the published SDK typings expose SnapshotNode.ref. If they do, making it optional is a source-breaking change for TypeScript consumers, and the repo has no changelog to record it.

Production wiring is not pinned yet. snapshotRefMinting(data) in buildSnapshotState has no test, and the web fixtures now give every node a ref. If someone removes that argument, the default dense mode returns and every test stays green. Please add a buildSnapshotState test with web agent-browser data that includes a ref-less node, and assert the node has no own ref key. Add an Android case that asserts dense numbering.

Simplicity question: could the rect-requiring resolution in @agent-device/selectors admit only addressable nodes and return winners and candidates typed SnapshotNode & { ref: string }? The touch-point competitor count, the keyboard-occlusion ref, the classification and evidence guards, and the unguarded prints would then be unnecessary or checked by the types. Read surfaces would keep the two kernel helpers. This needs an addressable-node type exported from kernel/snapshot and threaded through the resolution result. Is there a reason to prefer per-site guards? No ADR is needed either way.

Not blocking, and you can take or leave these: make the minting parameter of attachRefs required so each construction site states its mode. Widen snapshotRefBodies to readonly { ref?: string }[] and call it from interaction-touch-point.ts and src/mcp/tool-ref-pins.ts:181, which copy its body. Some sites still print an optional ref with no guard, such as target-visibility-stages.ts, resolution-disclosure.ts:87 and daemon/interaction/internal/find.ts:200 and :426. They are safe only because a ref-less node never carries a rect, and no type states that. The resolver change above would cover them.

CI is green, with 19 checks and none failing at ad579e0. There are no conflicts. The next step before merge is the design answer and the buildSnapshotState test.

Both earlier P3 threads are fixed at this head and can be resolved: the refAddress doc now covers both modes (#3547 (comment)), and the kind JSDoc now separates an absent ref from an absent kind (#3547 (comment)).

@thymikee
thymikee force-pushed the feat/snapshot-optional-refs branch from ad579e0 to 12f1280 Compare October 13, 2026 09:52
@thymikee thymikee changed the title feat(snapshot): make a node's ref optional and mint it per producer feat(snapshot)!: make a node's ref optional and mint it per producer Oct 13, 2026
@thymikee

thymikee commented Oct 13, 2026 •

Copy link
Copy Markdown
Member Author

Review round 3: the minting mode is pinned end to end, and two answers

Head 12f128010 (rebased on origin/main at 303eeb50a; #3515 and #3533 restacked on this).

Required: the production wiring is pinned

buildSnapshotState had no test for the snapshotRefMinting(data) argument — the observation was right, the web fixtures had all become ref-bearing, so deleting the argument returned every tree to dense numbering with the suite green. src/core/__tests__/snapshot-state.test.ts:682 now has one case per side of the mode:

  • agent-browser: three nodes, the middle one ref-less as the backend prints it (text on value, no rect), the outer two carrying e1 and e3. Asserts 'ref' in state.nodes[1] === false and that the published sequence is ['e1', undefined, 'e3'] — the second assertion is the one that matters for fix(web): preserve the backend ref so snapshot refs match actionable refs #2283: the producer's numbering is non-dense and stays byte-identical, so nothing can rename a ref out from under an action.
  • Android: three ref-less nodes through android-uiautomator, asserting ['e1', 'e2', 'e3'], i.e. dense numbering is what buildSnapshotState still does for a producer that mints nothing.

Mutation-checked: deleting the argument fails the agent-browser case alone (24 passed | 1 failed).

Design question: the addressable-node type — measured, and not in this PR

I tried it rather than reasoning about it. SnapshotAddressableNode = SnapshotNode & { ref: string } exported from kernel/snapshot, hasSnapshotRef narrowed to it, and resolveActionableTouchResolution / classifyActionableTouchCandidates made to admit and return only it. tsc -b on the full graph: 28 errors across 5 files — 18 in interaction-targeting.test.ts (its fixtures are SnapshotNode[]), 7 inside interaction-targeting.ts, and one each at selector-pipeline.ts:245, selector-action-resolution.ts:43 and the new buildSnapshotState test, the last three being the boundary where a caller holds a SnapshotNode from the engine or the pipeline.

Two findings, both against doing it as asked:

  1. Inside the resolver the guards come back. The winner is not the only node the resolution produces: the same-rect descendant, the hittable ancestor and the unverified wrapper control all arrive from a tree walk typed SnapshotNode. Silence-ing those 7 errors meant adding three hasSnapshotRef guards inside the resolver — the same class of guard the question wanted gone, moved from the presentation sites to the construction points. Net: relocated, not removed.
  2. It does not cover the four prints named in the review. target-visibility-stages.ts:45, resolution-disclosure.ts:87, find.ts:200 and :426 never call the resolver — each holds a node from SelectorPipelineOutcome / runNodePipelineStages. So the resolver's winner type cannot reach them; only the pipeline's outcome type can, and the same pipeline entries serve the read rows, whose entire answer in fix(web): put on-screen text in the snapshot tree instead of losing it #3515 is a ref-less node (find … list, wait text, get text). One outcome type cannot be both.

So the design is right, but the seam that has to carry it is the pipeline, not the resolver, and that is a second decision: rows that require a rect (act, findAct, actCoveredDiagnosis, cropTarget — already named in the table) get an entry that narrows its winner once and returns it typed addressable, refusing a ref-less winner with the row's own not-found reason; read rows keep the two kernel helpers. That makes the invariant those four prints lean on a checked fact instead of a comment, and the guards in target-evidence.ts, target-classification.ts and the settle tail fall out with it. It is not stack work: it edits the 12 production files that enter the pipeline plus the policy table's typing, while #3515 sits at 747 gross and #3533 at 927 of the same 1,000 budget. I have written the invariant into the type where the next reader will find it (packages/kernel/src/snapshot.ts:402, on the ref field: a ref-less node carries no rect, so no geometry-requiring path can reach one), and the pipeline split should land as its own PR on main after this stack — I'd rather not grow the middle layer of a three-PR stack to do it.

Answered: yes, the published typings expose it, so this is source-breaking

SnapshotNode.ref is in the shipped declarations: dist/src/sdk-contracts.d.ts carries the type and dist/src/index.d.ts re-exports it as type SnapshotNode, alongside SnapshotState.nodes: SnapshotNode[], CaptureSnapshotResult.nodes, and agent-device/contracts. So a consumer doing const r: string = node.ref, passing a SnapshotNode where ref is required, or indexing node.ref.length typechecks on the current release and fails on this one. examples/sdk/client-session.ts:68 is a live instance of exactly that shape — the SDK example needed target?.ref !== undefined before press({ ref: target.ref }), which is the break, demonstrated in-repo.

How the repo records one: there is no changeset or CHANGELOG — .changeset does not exist, CHANGELOG.md does not exist, and release.yml:203 builds notes with gh release --generate-notes --notes-start-tag, so the notes come from the commits since the last tag. The convention for a breaking change is therefore the ! in the conventional-commit subject, which lands in the squash title and so in the release notes — precedent dbf113a55 refactor(platform-android)!: report changed permission ids, not a joined string (#3023), whose body likewise stated that no changelog entry was possible. I have renamed this PR's title and its contract commit to feat(snapshot)!: … for the same reason, so the note carries the flag; the version bump itself stays with the release owner.

Take-it-or-leave-it, taken where it paid

  • snapshotRefBodies is widened to readonly { ref?: string }[] and the two copies deleted: interaction-touch-point.ts:55 now calls it on the competitor rects, tool-ref-pins.ts:181 on the published matches. One rule, one site.
  • attachRefs' minting stays defaulted. dense is the answer for every producer that exists and for any that mints nothing, so a call site that omits it is not unsaid-to — and all 85 call sites outside kernel (5 of them production: snapshot-state.ts, the iOS engine, the Maestro observation port, platform-android/snapshot.ts, test-kit/snapshot-builders) would state a mode the type already decides.
  • The unguarded prints are now covered by the invariant on the type (above) rather than per-site guards, pending the pipeline change.
  • Found while pinning the router path in fix(web): put on-screen text in the snapshot tree instead of losing it #3515: toDaemonFindData rebuilt the response with ref: match.ref, so a ref-less match left an own-property named ref whose value was undefined on the response data the CLI formatter, the MCP pin merge and a Node consumer read — an address-shaped absent, which is the one thing the optionality is for. (The wire form never showed it: JSON.stringify drops an undefined value. See the correction comment.) Fixed here (it is this PR's contract), with src/daemon/__tests__/selector-recording-find-data.test.ts pinning both the list arm and the text/attrs arm; each mutation fails its own case.

Both earlier P3 threads are already resolved at this head. pnpm check:affected --run --base origin/main result for this commit is in the body.

@cubic-dev-ai cubic-dev-ai Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All reported issues were addressed across 6 files (changes from recent commits).

Reply with feedback, questions, or to request a fix.

View guided diff | Turn on auto-fix | Re-trigger cubic

Comment thread src/daemon/selector-recording.ts Outdated
@thymikee
thymikee force-pushed the feat/snapshot-optional-refs branch from 12f1280 to 7a8f1ca Compare October 13, 2026 10:04
@thymikee

Copy link
Copy Markdown
Member Author

Correction to my round-3 comment

Cubic flagged (src/daemon/selector-recording.ts:66, now resolved) that one sentence I wrote was wrong, and it is: JSON.stringify drops a property whose value is undefined, so ref: match.ref never put a ref key on the wire — my "publishing a ref key holding null" was not true of --json output.

The defect is real but it is one layer in, and the fix stays. toDaemonFindData's return value is the response data object, not a string: the CLI formatter (output.ts:36 reads data.matches), the MCP ref-pin merge and a Node consumer of client.interactions.find all read that object, and there an own-property named ref whose value is undefined is exactly what the optional field exists to rule out. #3515's scenario assertion is 'ref' in matches[0] === false, an in-memory check, so it was red before this fix and is green after — that part of my account holds; what it proves is the own-property contract, not a JSON difference.

The comment now says that (a374f8a77, comment-only delta; the commit's own message was rewritten to match), and #3515's body carries the corrected sentence. Both of my round-3 comments are edited in place to say "own-property", and I have not added a claim about the JSON form anywhere else — grep -c 'ref: null' /tmp/* over what I posted is 0 after the edit.

Head is now 7a8f1ca44. The delta from 12f128010 is that one comment block plus the commit message: git diff 12f128010..HEAD --stat is 1 file changed, 5 insertions(+), 3 deletions(-), no behavior, no test; tsc -b and src/daemon/__tests__/selector-recording-find-data.test.ts re-run green on it, and the full check:affected gate is rerunning at this head under the machine lock, with the result added to the body. Downstream, #3515 (67f9bfe2a) and #3533 (c3329d93d) are tree-identical to the heads each last gated — git diff between old and new head for #3533 names only this comment file — so their gates still stand; #3515's reruns at the new base as a matter of course.

@thymikee
thymikee force-pushed the feat/snapshot-optional-refs branch from 7a8f1ca to 5f14d3c Compare October 13, 2026 10:44
@thymikee

Copy link
Copy Markdown
Member Author

This PR is ready at 5f14d3c. The earlier ref-address doc and test gaps from ad579e0 are fixed, and the optional ref contract now reads correctly in the code I checked.

Not blocking, and fine to take or leave: the five-line comment on daemonRefAddress in https://github.com/callstack/agent-device/blob/5f14d3c/src/daemon/selector-recording.ts#L64 could shrink to the contract (a match without a ref carries no ref key), and its three new test cases could move into the existing src/daemon/__tests__/selector-recording.test.ts instead of the sibling selector-recording-find-data.test.ts; also, the split of pipeline rows that require a rect into SnapshotNode & { ref: string } is deferred (https://github.com/callstack/agent-device/blob/5f14d3c/packages/kernel/src/snapshot.ts#L402), so a small follow-up issue linked from the PR body would keep it from getting lost.

The three cubic-dev-ai P3 threads no longer apply and can be resolved: the comment now describes the in-memory own-property contract (#3547 (comment)), the refAddress doc covers both minting modes (#3547 (comment)), and the kind JSDoc separates an absent ref from an absent kind (#3547 (comment)).

I ran no tests. I judged that the buildSnapshotState and toDaemonFindData tests fail without the change by reading the old code. I did not confirm that pruneGroupNodes and normalizeSnapshotTree keep the ref-less web static-text node that the ['e1', undefined, 'e3'] assertion depends on, though the author reports it green. The rule that a rect implies a ref for producers relies on #3515, which is not in this diff.

Checks at 5f14d3c are still queued or in progress, and none has failed. Typecheck & Package and Integration Tests cover the changed contract, so they need to finish green before merge. No code change is needed from this review.

@thymikee thymikee added the ready-for-human Valid work that needs human implementation, judgment, or maintainer merge label Oct 13, 2026
`SnapshotNode.ref` is declared as always present, which a capture whose producer resolves
actions against refs of its own cannot honor: a node the producer issued no ref for has one
missing, and a tree carrying that node would be published with `ref` present. The consumers that
offer a node as an address take the field at face value today — an overlay label, a pinned ref, a
settle tail row, a competitor listed in a touch-point refusal, a replay target's recorded ref
evidence — so the first capture that publishes such a node ships `@undefined` to an agent through
all of them.

`ref` is now optional and `attachRefs` takes the capture's minting mode. Dense producers
(Apple, Android, HarmonyOS, Linux) number their trees exactly as before; a producer that owns
its ref namespace (`backend-issued`, which agent-browser does — #2283) keeps its own refs and
mints nothing beside them. `hasSnapshotRef` and `snapshotRefBodies` are the one way to read
the absence, and every surface that offers a node as an address now reads it instead of
assuming one: overlay labels, the printed snapshot line, settle tails, MCP ref pins, the
touch-point competitor disclosure, keyboard occlusion, `find`/`get` reads, and the record- and
replay-time target classifiers, whose ref evidence a ref-less node cannot contribute.

No producer emits a ref-less node in this commit, so behavior is unchanged for every backend —
proof, not assertion: `git diff --name-only origin/main...HEAD | grep -c '^packages/platform-'` is
0, so no backend's capture path is touched here. The contract and its consumers land first so the
capture that needs it lands on top.
`refAddress` is the surface that decides whether a `find` result row prints an address, so it is
where a maintainer asks "when is this absent". Naming only the web case would invite reading the
field as a web quirk: dense producers mint a ref for every node through the same `attachRefs` call
and reach this helper with one present every time. The rule the code actually encodes is "read the
field, never infer a mode".

The `kind` JSDoc contrasted itself with `ref` as "required like `ref`", which was the old contract;
`ref` is optional now and for a reason, while `kind` is optional only as a fixture concession. The
two absences mean different things, so the sentence that compared them names the difference.
…t the platform

snapshotRefMinting asked whether a capture came from the web channel, but a channel
is not a producer: the provenance table itself says several producers can feed one
channel with different guarantees, and only agent-browser resolves actions against
refs of its own. Any second adapter that lands on the web channel — a webdriver- or
CDP-backed one — mints no refs and needs dense positional numbering, which the
channel check would have taken away silently, making half its tree unaddressable.

The mode now asks the owning question of the producer, and the parameter is the
whole provenance pair rather than a partial one so no caller can hand over a channel
and leave it to guess who acquired the tree. Behavior is unchanged for every
producer that exists today; the table is what the next producer has to reason about.
…er producer

The web ref-collision fix (#2283) rests on one argument at snapshot-state.ts:83, and no test reached it: deleting it returned every tree to dense numbering with the whole suite green, minting a ref onto web text its backend cannot resolve. One case per side of the mode now: an agent-browser capture publishes only the refs it issued (non-dense, byte-identical) and an Android capture stays densely numbered so every node remains addressable. Mutation-checked: removing the argument fails the first case alone.
…to a ref-less match

toDaemonFindData rebuilt the response with an unconditional `ref: match.ref`, so a match whose
capture issued no ref left an own-property whose value was not a ref on the response data. The
wire form never showed it (JSON.stringify drops an undefined value), but the data object is what
the CLI formatter, the MCP ref-pin merge and a Node consumer read, and an absent address is the
one thing the optional field exists to express. The builder now omits the key instead.
The touch-point competitor disclosure and the MCP ref-pin merge each restated the filter over their own shape. Typed over the field rather than over SnapshotNode, both call it. Also states on the ref field why the geometry-requiring paths print it bare: a ref-less node carries no rect, so no acting path can reach one.
@thymikee
thymikee force-pushed the feat/snapshot-optional-refs branch from 5f14d3c to e67b761 Compare October 13, 2026 12:49
@thymikee

Copy link
Copy Markdown
Member Author

Took the non-blocking items at e67b761af (new head; the stack moved because main picked up two more merges while you were reviewing — #3567/#3568, files disjoint from this stack):

  • daemonRefAddress comment — shrunk to the contract: ref-less match carries no ref own-property, the wire form already hid it, the response DATA object is where the optional field has to hold. The "two consumers read one rule" sentence went; the type says it.
  • Follow-up issue — filed Type the rect-requiring resolution rows as ref-bearing instead of proving it by comment #3576 and linked from the PR body. It records the measured cost of the deferred narrowing (28 tsc errors across 5 files for the resolver/classifier-typed attempt), why the guards relocate inward, and that the narrowing has to sit at the pipeline entry because the four unguarded prints are fed by the pipeline outcome, not the resolver.
  • Test-file move — declined, with a reason: src/daemon/__tests__/selector-recording.test.ts declares itself in its header as the file for recordIfSession's observation-only classification (ADR 0012 amendment). toDaemonFindData's ref-key contract is a different domain question, and the repo's module rule is that a file is named for the question it answers, so the sibling file stays. Happy to move them if you'd rather have one file anyway — it's mechanical.

On your two verification notes, to make the record honest: pruneGroupNodes/normalizeSnapshotTree do keep the ref-less static-text node — the ['e1', undefined, 'e3'] assertion is green locally (src/core/__tests__/snapshot-state.test.ts, 25 tests), and mutation-wise, deleting the snapshotRefMinting(data) argument at snapshot-state.ts:83 mints e2 and fails exactly that web case. And yes — the rect-implies-ref rule for the acting paths depends on #3515's capture; #3547 alone leaves the tree shape unproven, which is why the merge order is 3547 → 3515 → 3533 and #3576 lists #3515 as a dependency.

Both edits are comment-only against 5f14d3c, so nothing you reviewed changes behavior. pnpm check:affected --run --base origin/main re-run green on this head; CI is running.

@thymikee

Copy link
Copy Markdown
Member Author

Thanks for the update. The earlier feedback is addressed at e67b761: the three cubic-dev-ai doc threads are fixed at head, and the code has no blocking problems. CI is green, with 19 checks reported and none failing. There are no conflicts.

Not blocking, so take it or leave it: src/core/interactors/web.ts:61 (https://github.com/callstack/agent-device/blob/e67b761/src/core/interactors/web.ts#L61) stamps producer 'agent-browser' on every snapshot, whatever WebProvider it wraps. snapshotRefMinting keys on that producer, so a second web adapter would get 'backend-issued' and lose refs on nodes it left unaddressed, which is the channel-keyed outcome the PR body says it avoids. The emulated provider in test/integration/provider-scenarios/web-world.ts hit this and needed refs e1 to e6 added. The kernel test that claims a second producer keeps dense numbering uses a cast pair that never reaches this route. Should WebSnapshotResult carry the provider's own producer, so the producer-keyed decision holds at that route? If not, please tone down the PR body claim and the kernel test comment to say the guarantee depends on web.ts stamping the producer correctly.

The three cubic-dev-ai P3 threads are fixed at head, so you can resolve them: selector-read.ts doc (#3547 (comment)), kernel snapshot.ts:415 (#3547 (comment)), and daemonRefAddress JSDoc (#3547 (comment)).

I did not run tsc or the tests locally, so the code check rests on green CI. The four acting-path prints of node.ref stay safe only because a rect-bearing node always has a ref. That fact is documented, not typed, and #3576 tracks typing it. Nothing else stands in the way of a maintainer merge decision.

@thymikee
thymikee added this pull request to stack #3582 October 13, 2026 14:24
@thymikee
thymikee merged commit 7462836 into main Oct 13, 2026
19 checks passed
@thymikee
thymikee deleted the feat/snapshot-optional-refs branch October 13, 2026 14:24
@github-actions

Copy link
Copy Markdown
Contributor
PR Preview Action v1.8.1
Preview removed because the pull request was closed.
2026-10-13 14:32 UTC

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-human Valid work that needs human implementation, judgment, or maintainer merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant