Skip to content

fix(computer-use): read elements reported at infinity without a frame instead of crashing the helper - #10034

Merged
waleedlatif1 merged 1 commit into
stagingfrom
fix/computer-use-nonfinite-frames
Nov 14, 2026
Merged

waleedlatif1 merged 1 commit into
stagingfrom
fix/computer-use-nonfinite-frames

Conversation

@waleedlatif1

Copy link
Copy Markdown
Collaborator

Summary

  • Some apps report hidden Accessibility elements at infinity (Maps does once its search field has focus). The helper copied that position into its reply, and JSONSerialization raises an Objective-C exception on a non-finite number, which Swift cannot catch, so the helper aborted and every later get_app_state on that app crashed it again
  • axRect now treats a non-finite position or size as no frame, so those elements come back without coordinates and the rest of the state is read normally
  • encodeReply checks JSONSerialization.isValidJSONObject first and throws reply_unencodable, which the main loop sends as an error reply, so no value can abort the helper again

Type of Change

  • Bug fix

Testing

  • Live fixture gains an element whose accessibilityFrame is at infinity; tests/live.py asserts it comes back without a frame. Passes on this branch's helper; with the pre-fix helper the same test aborts at the first get_app_state with Invalid number value (infinite) in JSON write
  • Native tests cover axRect with finite and infinite values and encodeReply refusing a non-finite number (pass)
  • Real Maps in a macOS VM: focusing the search field then get_app_state (with and without screenshot) now succeeds, and typing into the field afterwards works with the helper still running; before, the helper aborted every time (the pre-batch helper did too, so this predates the recent computer-use changes)

Checklist

  • Code follows project style guidelines
  • Self-reviewed my changes
  • Tests added/updated and passing (new tests pass the test-audit authoring gate)
  • No new warnings introduced
  • I confirm that I have read and agree to the terms outlined in the Contributor License Agreement (CLA)

@vercel

vercel Bot commented Nov 14, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated
docs Skipped Skipped Nov 14, 2026 2:37am UTC

Request Review

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 5 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

Turn on auto-fix | Re-trigger cubic

@greptile-apps

greptile-apps Bot commented Nov 14, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5

[Medium impact] The PR appears safe to merge.

Summary

The helper now reads Accessibility elements with non-finite frames without including their coordinates.

  • App-state reads keep running when apps report infinite element frames.

Reviews (1) · Last reviewed commit: "fix(computer-use): read elements reporte..." · Reviewed by Greptile

@waleedlatif1
waleedlatif1 merged commit abd2e8d into staging Nov 14, 2026
50 checks passed
@waleedlatif1
waleedlatif1 deleted the fix/computer-use-nonfinite-frames branch November 14, 2026 02:43

This branch was previously deployed

1 inactive deployment
Preview — aed69159 Deployed Nov 14, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant