Skip to content

fix(desktop): capture headed browser tabs while macOS is locked - #27092

Open
mhodgson wants to merge 2 commits into
pingdotgg:mainfrom
mhodgson:codex/fix-macos-locked-browser
Open

mhodgson wants to merge 2 commits into
pingdotgg:mainfrom
mhodgson:codex/fix-macos-locked-browser

Conversation

@mhodgson

Copy link
Copy Markdown

Problem

On a Mac Studio hosting T3 Code, locking macOS causes agent browser snapshots to time out and a remote desktop client's browser panel to remain on Connecting. Unlocking restores operation. The failure reproduces with the host displays still on and with a Screen Sharing connection present.

Host: macOS 27.0.1, Apple Silicon. Observed on 0.1.4-nightly.20261028.3824 (eb751fa0730d), Electron 44.9.3 / Chromium 152.0.7977.167. This build is newer than the hidden-guest capture fix in #25184, which closed #24514 and #22461. This PR addresses the remaining locked-macOS case. The unanswered initial capture in #24852 is related, but this PR does not claim to resolve every cause of that failure.

Change

On macOS, keep server-driven desktop guests inside the window behind the app, including between captures, and use Electron's webContents.capturePage for main-page viewport PNG/JPEG captures. In isolated Electron tests, keeping the guest inside the window is necessary for composition, and native capture returns unchanged frames when CDP Page.captureScreenshot stalls under screen lock.

The native path retains JPEG quality. Requests with clipping, capture beyond the viewport, fromSurface: false, WebP, or unknown options continue through CDP. Child target sessions continue through CDP. The existing screenshot queue, focus leases, and capture lifecycle remain in place. Other host platforms keep their existing rendering and capture paths. No wire contract changes.

This deliberately keeps inactive server-driven macOS guests inside the viewport. That can increase background rendering work compared with moving them off-window between captures; this PR has no quantitative energy-use measurement. Ordinary local tabs retain the existing suspension behavior.

Reproduction

  1. Run the desktop app on a Mac host and connect from another Mac's T3 desktop app.
  2. Have the agent open a browser tab and confirm snapshots and remote viewing work while the host is unlocked.
  3. Lock the host with its displays still on.
  4. Request a snapshot and open the remote browser panel. The affected build times out and the viewer stays on Connecting.
  5. Unlock the host; browser operation recovers.

Validation

  • vp run check:changed: format, lint, knip, affected package typechecks, 37 focused desktop tests, and 2 React Compiler checks passed.
  • New tests cover native capture with unavailable CDP capture, JPEG quality, forwarding unsupported capture options, and macOS-only selection of the native path. Existing screenshot lifecycle tests continue to pass.
  • vp run build:desktop passed for the candidate used in the packaged desktop check.
  • An isolated Electron 44.9.3 headed, sandboxed webview using the changed capture helper and production input helper captured repeated frames and accepted input while macOS was locked.
  • An isolated packaged T3 desktop candidate using the changed macOS capture/rendering behavior streamed 12 frames through /api/preview-stream/ws. The fixture observed a mouse click and the entered text locked-input-test. macOS reported the screen locked at both ends of the test. The tab used the desktop renderer, with no headless fallback.
{"lockedAtStart":true,"buttonClicked":true,"inputValue":"locked-input-test","frames":12,"lockedAtEnd":true,"passed":true}

The stream check used a protocol client against the actual T3 viewer WebSocket. The patched UI on a second physical Mac has not been verified. The packaged check used Electron 44.10.0 and the same macOS behavior before the added platform guards; the final guards leave that macOS path unchanged. Initial app startup while already locked is outside the verified scope. No installed app or live user data was modified.

Before: capture timed out and the remote viewer produced no frame. After, this frame came from the locked-host interaction check:

Patched headed T3 browser after receiving a click and text while the Mac host stayed locked

Structured Codex autoreview (--mode local --prompt-file ...): no accepted/actionable findings.

Authored with Codex (gpt-6.1-sol) in T3 Code.

@github-actions github-actions Bot added vouch:unvouched PR author is not yet trusted in the VOUCHED list. size:M 30-99 changed lines (additions + deletions). labels Oct 29, 2026
@macroscopeapp

macroscopeapp Bot commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

Approvability

Verdict: Approved at d4e4a67

Macroscope's review found this PR approvable — This restores macOS browser-tab screenshots while locked using native guest capture and keeps server-driven tabs paintable behind the app. Capture remains scoped to existing guest tabs, with other platforms and unsupported screenshot options retaining their existing paths.

You can add or adjust custom eligibility rules. Learn more.

@coderabbitai

coderabbitai Bot commented Oct 29, 2026

Copy link
Copy Markdown
Contributor

Review in Change Stack →

No actionable comments were generated in the recent review.

Recent review info
Run configuration
Commits

Reviewing files that changed from the base of the PR and between 08a463e and d4e4a67.

Files selected for processing: 6
  • apps/desktop/src/preview/DesktopBrowserHost.test.ts
  • apps/desktop/src/preview/DesktopBrowserHost.ts
  • apps/desktop/src/preview/GuestCapture.test.ts
  • apps/desktop/src/preview/GuestCapture.ts
  • apps/web/src/browser/HostedBrowserWebview.tsx
  • apps/web/src/browser/hostedBrowserWebviewStyle.ts

Included review availability: This review used your included allowance. Your plan provides up to 10 included reviews per hour; 9 remain after this review.

Walkthrough

On macOS, server-driven preview guests use a visible, non-interactive wrapper fallback, and eligible top-level screenshot commands use native page capture. Requests with unsupported capture parameters continue through CDP. Tests cover platform routing and native/CDP capture behavior.

Changes: Preview capture

Layer / File(s) Summary
Keep server-driven macOS guests visible
apps/web/src/browser/HostedBrowserWebview.tsx, apps/web/src/browser/hostedBrowserWebviewStyle.ts
The webview passes a macOS-gated serverDriven value to the wrapper-style resolver. The resolver applies the visible, non-interactive fallback when renderingActive or serverDriven is true and the active-rectangle branch does not apply.
Route eligible screenshots through native capture
apps/desktop/src/preview/DesktopBrowserHost.ts, apps/desktop/src/preview/GuestCapture.ts, apps/desktop/src/preview/GuestCapture.test.ts, apps/desktop/src/preview/DesktopBrowserHost.test.ts
DesktopBrowserHost routes sessionless macOS Page.captureScreenshot commands through captureGuestPage. The helper uses native capture for supported parameters and forwards other requests to CDP. Tests cover capture parameters and platform routing.

Priority: Normal

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix · Severity of issue fixed: Medium

Possibly related issues

  • pingdotgg/t3code#23022 — The changes keep server-driven macOS guests inside the app window and use native capture for eligible screenshots.
  • pingdotgg/t3code#23936 — The changes address hidden preview capture by keeping server-driven macOS guests inside the app window; child-target sessions still use CDP.
  • pingdotgg/t3code#23139 — The wrapper change addresses the non-paintable guest condition on macOS; the Windows path remains unchanged.
  • pingdotgg/t3code#17483 — The changes apply the paintability approach to server-driven macOS guests.

Possibly related PRs

  • pingdotgg/t3code#24133 — This PR and #27092 modify desktop guest screenshot preparation and capture.
  • pingdotgg/t3code#9916 — Both PRs modify top-level Page.captureScreenshot handling in DesktopBrowserHost and use native capture.
  • pingdotgg/t3code#24515 — Both PRs change the hidden desktop preview screenshot path and its rendering behavior.

Suggested labels: bugfix, size:M

Suggested reviewers: shivamhwp, maria-rcks, macroscopeapp

Merge Risk: Minimal · up to d4e4a

Eligible macOS screenshots use native capture; no merge-blocking issue was established.

Pre-merge checks: Passed 3 · Inconclusive 1

Failed Failed checks: 1 inconclusive

Check name Status Explanation Resolution
Description check Inconclusive The description follows the template structure. It has Problem, Change, Scope and approval, Verification, and an agent attribution line. The Problem section gives the symptoms, the affected build, and… Add a link to the triaged bug issue or maintainer-approved discussion for this change, including the approval comment. If no prior issue exists, explain why this qualifies as a very small, focused fix of an obvious bug. Keep the existing re…
Passed Passed checks: 3 passed
Check name Status Explanation
Title check Passed The title names the desktop app, the headed browser tab capture, and the macOS locked condition. The raw summary shows the change keeps macOS server-driven guests in the window and routes main-page vi…
Linked Issues check Passed No active directly linked issues remain. Issue #24514 is closed and provides historical context only. Therefore, this pull request has no linked-issue coding requirements to assess.
Out of Scope Changes check Passed The changes stay within the current pull request intent. They address macOS guest rendering and capture behavior, preserve CDP fallbacks and non-macOS paths, and add focused tests. No unrelated change…
Full details: Title check

Explanation

The title names the desktop app, the headed browser tab capture, and the macOS locked condition. The raw summary shows the change keeps macOS server-driven guests in the window and routes main-page viewport captures through Electron's native capturePage, which matches the title's focus on capturing while locked. The title is specific and concise, and it follows the conventional commit format from the template.

Full details: Out of Scope Changes check

Explanation

The changes stay within the current pull request intent. They address macOS guest rendering and capture behavior, preserve CDP fallbacks and non-macOS paths, and add focused tests. No unrelated change is demonstrated.

Full details: Description check

Explanation

The description follows the template structure. It has Problem, Change, Scope and approval, Verification, and an agent attribution line. The Problem section gives the symptoms, the affected build, and reproduction steps. The Change section explains the macOS-only routing and the fallback to CDP. The Verification section lists focused tests, a build check, isolated Electron checks, a packaged check with a JSON result, and a before/after image. The Scope and approval section is the weak point. It cites issues #24514, #22461, and #24852 as context, but it does not link a maintainer-approved triage issue or discussion for this fix, and it does not explain why a prior approval is not needed. The Verification section also reports that the patched UI on a second Mac was not verified, which is honest disclosure, but the template asks for stated limits, and this is present.

Resolution

Add a link to the triaged bug issue or maintainer-approved discussion for this change, including the approval comment. If no prior issue exists, explain why this qualifies as a very small, focused fix of an obvious bug. Keep the existing reproduction, verification, and limitation notes.

Finishing Touches
🧪 Generate unit tests (beta)
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

maria-rcks commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Note

Written by gpt-6.1-sol on behalf of Maria

Verified in a real dev environment

linux regression checks passed at d4e4a67b8863a3c2ea18c07544594be32d7eebbc. the macos lock fix remains unverified.

  • read the description, discussion, reviews, six-file diff, and both commits. the pr is open; neither commit is by juliusmarminge or t3dotgg.
  • started github/main (e00ebb6e2641a2681d7612bccc5d8810403d99b8) in ../main after vp i, then stopped its captured process group before starting the exact pr head after another vp i. both used isolated empty state, with seeding disabled. only one verification dev server ran at a time.
  • drove the real web ui with the repo's playwright and /usr/bin/google-chrome. added a project pointing at the real checkout, created a thread, selected codex gpt-6.1-sol, and submitted real provider turns through the local proxy. no mocks, fixtures, replay provider, or source edits.
  • on main and the pr, the agent opened an independent server browser tab, saved a 1280×800 png through preview_snapshot, clicked "learn more", reached https://www.iana.org/help/example-domains with http 200, returned, and captured another snapshot.
  • on the pr, two successive additional saved snapshots succeeded. revealed that same agent-created tab, took control through the web client, saw its streamed frames, clicked the link inside the streamed page, then used back and refresh. this exercises capture, navigation/input, and the viewer through the real client/server/provider path.

short runtime transcript, read from the isolated servers' persisted tool results:

main e00ebb6e26: model=gpt-6.1-sol; provider turns=completed
preview_open -> https://example.com/
preview_snapshot -> image/png 1280x800
preview_act click -> https://www.iana.org/help/example-domains
preview_navigate -> https://example.com/
preview_snapshot -> image/png 1280x800

pr d4e4a67b88: model=gpt-6.1-sol; provider turns=completed,completed
same browser flow -> completed
preview_snapshot (two additional captures) -> image/png 1280x800, image/png 1280x800
t3_preview_list -> runtime=server
web viewer click -> https://www.iana.org/help/example-domains
web viewer back -> https://example.com/

main linux baseline, with the same destination as the pr capture:

main linux browser viewer after a real codex turn

pr linux browser viewer:

pr linux browser viewer showing the same page after repeated captures

pr recording of input into the streamed browser, navigation, back, and refresh (animated copy for github rendering):

pr linux streamed browser navigation recording

inspected the captures and played the uploaded recording, including seeking. github reports 24 passed checks and 8 skipped, zero failing or pending; git.cafe groups these as 32 successful. git.cafe renders these external media as links; the uploaded video was opened and its playback and seeking checked in the t3 browser. no code was changed or pushed; no additional build/test suite was run.

Not verified

the original bug could not be reproduced on this linux vm. both production changes are gated to macos: electron's native webContents.capturePage path and keeping server-driven macos guests inside the app window. the browser used above explicitly reports runtime=server; it does not execute the changed electron code. these linux captures are regression evidence, not before/after proof of the lock fix.

macos host locking, repeated unchanged frames while locked, a remote desktop client's viewer while its mac host is locked, native jpeg quality and cdp fallback under lock, and startup while already locked still require a real macos desktop host. windows and native ios/android clients were not exercised. no claim is made that this pr's macos fix is verified or ready to merge from this run.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:M 30-99 changed lines (additions + deletions). vouch:unvouched PR author is not yet trusted in the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: off-window Electron preview guests stall snapshots and block later captures

2 participants