Skip to content

[Bug]: iOS remote preview — "You have control" but taps never reach the tab; streams stuck on "Connecting to browser…" after refresh / for other tabs #19420

Open
#19456
Open
[Bug]: iOS remote preview — "You have control" but taps never reach the tab; streams stuck on "Connecting to browser…" after refresh / for other tabs#19420
#19456
Labels
via-triageFiled through npx t3 triage

Description

@zachtjohnson01
zachtjohnson01
opened on Oct 14, 2026

Summary

From the iOS app, a desktop preview tab can't be driven: after "Take control" the phone shows "You have control", but taps/scrolls on the page never reach the tab on the Mac. Toolbar actions do (Refresh reloads the page). After that refresh, and for every other preview tab in the same thread, the phone stays on "Connecting to browser…" indefinitely while the tabs are alive on the Mac.

Environment

  • T3 Code desktop (macOS, Apple silicon) 0.0.46-nightly.20261013.3080, paired with the iOS app (latest nightly at the time).
  • Three agent-opened preview tabs (via preview_open) in one thread, each a static local page (http://127.0.0.1:<port>/tabN.html) that logs every pointerdown / click / keydown / input / wheel / scroll / focus event into the page.

Steps

  1. Agent opens 3 preview tabs in a thread (default "fill" viewport, 1280×800).
  2. On iPhone, open the thread → open tab 3 → tap Take control ("You have control" appears; the stream shows the page).
  3. Tap buttons, type, scroll the page and inner scroll boxes.
  4. Tap Refresh in the phone toolbar.
  5. Open tab 1 and tab 2 on the phone.

Expected

Taps/scrolls reach the page; the stream keeps working after a refresh; other tabs connect.

Actual

  • Step 3: the page logged 0 pointer/click/key/wheel/scroll events; the desktop trace has no PreviewManager.handleHumanInput spans in that window (it normally logs many when driving from the desktop). preview_status still reported control.owner: "agent" afterwards (generation had moved to 4, so control handoffs happened, but no input arrived).
  • Step 4: the Refresh did reach the Mac (the page logged a fresh load), control went back to owner: "agent", generation: 0, and the phone then stayed on "Connecting to browser…".
  • Step 5: tabs 1 and 2 never got past "Connecting to browser…" (tab 1 also had a 430×932 device preset; tab 2 was plain fill, same result). Both pages were alive on the Mac (preview_evaluate worked).
  • Separately, the one stream that did work was very blurry: the 1280×800 tab at DPR 1 is scaled ~3× down to the phone width.

Notes

Activity

  1. juliusmarminge commented on Oct 14, 2026

    @juliusmarminge
    juliusmarminge
    on Oct 14, 2026
    Member

    Note

    Grok responding on behalf of Julius.

    We can't confirm this as a bug from the code yet, so we're marking it as needing more info.

    • Input: PreviewManager.handleHumanInput only fires for input on the desktop's own preview webview. It doesn't fire for input arriving through the remote stream, so having no spans there doesn't show that phone taps were dropped. We need the server-side logs from the stream socket for that window.
    • Stuck "Connecting to browser…": open PR fix(preview): keep one browser page across thread switches #18873 reworks how desktop preview pages are streamed to other clients. It adds frame leases for hidden native tabs, which matches tabs that aren't in front on the Mac never producing frames. Please retest once it lands on nightly.
    • Blurry stream: the 1280×800 DPR-1 viewport scaled down to phone width is how it works today. Please open a separate Ideas discussion if you want it changed.

    Questions:

    1. Was the thread open and visible in the Mac app while you used the phone?
    2. Can you attach the server log from that window? Look for preview stream connect/close lines.
    3. Does a phone-only thread (no desktop window open) behave the same way?
  2. juliusmarminge
    added
    needs more infoInitial triage showed no bug. Awaiting more info
    via-triageFiled through npx t3 triage
    on Oct 14, 2026

  3. mxdhavgautam commented on Oct 15, 2026

    @mxdhavgautam
    mxdhavgautam
    on Oct 15, 2026
    Contributor

    Note

    🤖 Claude Opus 5.5 responding on behalf of Madhav

    We hit the input half of this on Linux, from a remote viewer of a tab the desktop app draws, and traced it to a missing viewport message. I reproduced it with the web client. The mobile client has the same gate, so iOS should behave the same way. This doesn't cover the stuck "Connecting to browser…" part.

    Cause

    1. The server drives a desktop tab through chromium.connectOverCDP (connectDesktopPage). Playwright never sizes that page, so tab.page.viewportSize() is null.
    2. broadcastViewport returns early when layoutSize is null, so a viewer of a desktop tab never gets a viewport message. The viewer's own resize is ignored for desktop tabs.
    3. Without a viewport, pagePoint returns null on web and mobile, so the client never sends mouse, touch or wheel input. "You have control" still shows because control doesn't depend on the viewport. Refresh and the other toolbar actions work because they don't use coordinates.

    This is the same viewportSize() === null gap behind #16690 and #18593. #19913 adds readPageMetrics for snapshots, which reads the size a viewport message needs.

    Reproduction on main 42da166d13

    vp run dev:desktop with a throwaway T3 home, Linux. An agent called preview_open on a local page that logs every pointerdown, click and wheel. The thread was open in the desktop window, so the desktop drew the tab (1280x800, devicePixelRatio 1). A second browser paired as a web client opened the same tab, took control, clicked the page twice and scrolled. Its preview stream socket:

    RECV {"type":"control","canOperate":true,"controller":"agent","generation":0,"dialog":null}
RECV <first frame>
SENT {"type":"takeControl"}
RECV {"type":"control","canOperate":true,"controller":"you","generation":1,"dialog":null}
SENT {"type":"resize","width":539,"height":775}

    No viewport message arrived, the client sent no mouse or wheel messages, and the page logged 0 events.

    To confirm that's the cause, I made a local change (not a PR). When a viewer attaches to a desktop tab, the server reads Page.getLayoutMetrics and pushes that size as a viewport message. With the same steps, the socket got {"type":"viewport","width":1280,"height":800}, the client sent mouse and wheel messages, and the page logged trusted pointerdown, pointerup and click at (216, 152) for each click, plus a wheel. (216, 152) is the point that was clicked on the stream.

    The same null shows up in preview_resize, as #18593 notes. In a separate run on a nightly desktop build, preview_resize to a freeform 560x800 returned viewport: {width: 1280, height: 800}, the UNATTACHED_FILL_VIEWPORT fallback, while innerWidth in the page was 560.

    On your questions: this needs the desktop to draw the tab, so the thread has to be open in the desktop window (your question 1). A tab with no desktop window runs headless with a Playwright viewport and should get a viewport message (question 3). I didn't test that case. The socket log above stands in for the server log (question 2).

    A fix looks small and server-only: publish the desktop page's own size to viewers, and publish it again when the size changes. Happy to send a PR if that direction works for you.

  4. LeighAtkins commented on Oct 21, 2026

    @LeighAtkins
    LeighAtkins
    on Oct 21, 2026

    Reproduced on v0.1.1 stable with a different client/host pair: Mac desktop app → Windows 10 host (the desktop app's own local server, Windows 10 Home 19045).

    For any tab hosted on the Windows machine, Take control on the Mac succeeds. The host's preview_status shows control: { owner: "human", generation: 3 }. But no clicks, scrolls or typing from the Mac reach the page. This happens on an empty tab, on google.com, and on a signed-in helpdesk page. The same tabs work normally from the PC's own panel. The Mac view is also a blurry stream.

    The host tabs are drawn by the Windows desktop app (page UA T3CodeBeta/0.1.1 … Chrome/152.0.7977.130 Electron/44.4.5), which matches the missing-viewport cause traced above. #19456 isn't merged, so 0.1.1 doesn't include the fix.

    The other direction fails too. A tab opened on the Mac works there, but the host agent can only list it: t3_preview_list returns it (no runtime field), and preview_snapshot({ tabId }) returns PreviewAutomationTabNotFoundError (#16901). So no tab can be used by both the remote user and the agent.

    Practical impact: you can't sign in to a site from the remote machine for the agent to use. The login has to be done physically at the host. Once it's done there, agent tabs on the host reuse it.

    Posted by Claude Code (Opus 5.5) on behalf of @LeighAtkins.

  5. genomez commented on Oct 21, 2026

    @genomez
    genomez
    on Oct 21, 2026 – with ChatGPT Codex Connector

    I can reproduce closely related mobile input failures with an Android viewer and a Windows desktop host.

    Environment

    • Desktop host: Windows 11 x64
    • T3 Code: 0.1.2-nightly.20261021.3486
    • Android app: 2.0.0, build 350

    Reproduction

    1. An agent opens a login page using preview_open.
    2. View that tab in the Android app; the page is visible.
    3. Tap "Take control".
    4. Attempt to interact with the login form.

    Observed behavior

    • Android interaction does not work: I cannot tap/type into the page.
    • After attempting Take control, preview_status still reports owner: "agent".
    • The control generation increased from 0 to 4.
    • Taking control of the same tab from the Windows desktop works, and I could sign in there normally.

    The generation change resembles the evidence already in this issue. However, I have not established whether ownership immediately reverts, whether input is dropped despite a successful handoff, or whether another transition is responsible.

    Related proposed fix

    #19456 appears relevant because it addresses missing viewport information and dropped mobile input for desktop-hosted tabs. I have not tested that patch.

    I also have a separate Windows reproduction where completely covering the T3 window stalls snapshots and Android viewing. This comment concerns the input failure when the page is already visible on Android; I am not assuming the two failures share one cause.

  6. juliusmarminge
    removed
    needs more infoInitial triage showed no bug. Awaiting more info
    on Oct 22, 2026

  7. WhiteWarrior625 commented on Oct 23, 2026

    @WhiteWarrior625
    WhiteWarrior625
    on Oct 23, 2026

    Note

    Posted by Claude Code (Opus 5.5) on behalf of the account owner.

    This reproduces with a desktop client, not only mobile. The host is the Linux desktop app on 0.1.2-nightly.20261022.3541. The client is the Windows desktop app on the same build, connected to the Linux host as a remote environment.

    1. An agent on the host runs preview_open on a long static page (http://127.0.0.1:8741/page.html). The host's desktop.trace logs PreviewManager.createTab in the same second, so the host desktop app draws the tab.
    2. On the Windows client, open that thread's preview panel and click Take control. The panel shows "You have control".
    3. Use the mouse wheel over the page, or click a link.
    • Expected: the page scrolls, and the click lands.
    • Actual: nothing happens. Address-bar navigation and Reload still work.

    The cause is the one the earlier comment here traced, confirmed live on 3541:

    • preview_status for a host-drawn tab has no viewport field. My test tab returned viewportSetting: {"_tag":"fill"} and nothing else, so tab.page.viewportSize() is null. Playwright's connectOverCDP creates the context with noDefaultViewport: true. The call is in ServerBrowserContexts.ts:187.
    • layoutSize() returns that null (ServerBrowser.ts:600-602), so broadcastViewport() returns before it sends anything (ServerBrowser.ts:624-628). The viewer's own resize is ignored for desktop tabs (ServerBrowser.ts:2201).
    • On the client, pagePoint() returns null while viewportRef is unset (apps/web/src/browser/ServerBrowserSurface.tsx:265-269). The wheel listener then returns before preventDefault or queueWheel (ServerBrowserSurface.tsx:474-486), and pointer input takes the same path.
    • From the same tab, the agent's preview_scroll moves the page (scrollY 0 to 600). The server-side wheel dispatch (ServerBrowser.ts:2147-2157) works. The client never sends it.

    #19456 looks like the right fix. Please also run its test matrix with a desktop client against a Linux desktop host.

  8. artjom-mashanin commented on Oct 23, 2026

    @artjom-mashanin
    artjom-mashanin
    on Oct 23, 2026

    Note

    Posted by Claude Code (Opus 5.5) on behalf of the account owner.

    Another data point, on the newest nightly: macOS host → macOS client. The host is the macOS desktop app on 0.1.3-nightly.20261023.3579 (its own local server). The client is the macOS desktop app on a second Mac, connected via T3 Connect.

    Same symptom: Take control succeeds, then clicks and typing do nothing.

    I checked the server side with non-pausing logpoints on the viewer input handler (ServerBrowser.ts:2415) and on dispatchViewerInput:

    • Clicking Take control logs one message from the remote viewer: {"type":"takeControl"}, with canOperate=true and a desktop-hosted tab. Control passes to that viewer (owner = viewer, epoch 0 → 1).
    • After that, clicking and typing in the page logs nothing at all. No mouse, wheel or key message reaches the server, so dispatchViewerInput never runs.

    So the input is dropped on the client before it is sent, as traced above: the host never sends a viewport message for desktop-hosted tabs, so pagePoint() stays null. #19456 still looks like the fix. It also applies to macOS hosts.

  9. sprintthunder-dotcom commented on Oct 24, 2026

    @sprintthunder-dotcom
    sprintthunder-dotcom
    on Oct 24, 2026 – with Cursor

    Same issue here. iPhone T3 Code connected to my desktop T3 Code. The remote browser stays stuck on "Connecting to browser…" (in my case on a Microsoft login page), and I'm not near the computer to open the browser panel. +1, waiting for a fix (#19456 / #22877). Thanks!

  10. illumin3dpb-ship-it commented on Oct 27, 2026

    @illumin3dpb-ship-it
    illumin3dpb-ship-it
    on Oct 27, 2026 – with ChatGPT Codex Connector

    Retested the input half after updating the Fedora desktop host from 0.1.3-nightly.20261026.3717 to 0.1.4-nightly.20261027.3770. The remote MacBook viewer still shows the page and “You have control” / “Release control”, but clicks and typing do nothing. The owner also reports the failure through the direct T3 connection, outside the surrounding Relay.

    Read-only inspection of the installed 3770 AppImage still finds the missing-viewport path described above:

    • connectDesktopPage uses chromium.connectOverCDP(endpoint, { timeout: 15000 }).
    • layoutSize returns page.viewportSize() for a desktop tab.
    • broadcastViewport returns without sending anything when that value is null.
    • The installed web viewer’s coordinate mapper requires a viewport even when a frame exists and the controller is you.

    I extracted those server functions, the exact installed viewer coordinate mapper, and bundled Playwright 1.60.0, and ran them against an owned, isolated Chromium/CDP fixture. Results on 3770:

    Real JPEG frame:              4,956 bytes
Playwright desktop viewport:  null
Viewport messages emitted:    0
Controller:                   you
Mapped click:                 null
Measured CSS viewport:        780 x 493
Same click with measured size: (390, 246.5)

    This is a source-path reproduction, not a packet capture of the owner’s MacBook socket or a full Electron end-to-end test. No installed T3 code or settings were changed. It adds current-build confirmation for #19456; the first-frame fix discussed in #22461 does not resolve this input path.

    Posted by Codex on behalf of the account owner.

  11. lechatlisse commented on Oct 29, 2026

    @lechatlisse
    lechatlisse
    on Oct 29, 2026

    Posted by Claude Code (Opus 5.5) on behalf of the account owner.

    Same on a macOS client. Host: Mac Studio, T3 Code (Nightly) 0.1.4-nightly.20261028.3838, server run by the desktop app, screen unlocked. Client: T3 Code desktop on a MacBook Air, connected over T3 Connect.

    Repro

    1. The agent opens http://127.0.0.1:54323/project/default (a local Supabase Studio) with preview_open. The stream shows on the Air.
    2. On the Air, press Take control. The label reads "You have control".
    3. Click a sidebar link in the page ("Table Editor") and hover over icons.

    Result: nothing happens, and hover has no visible effect. Afterwards, preview_status shows the URL unchanged at /project/default, and the control generation went from 0 to 2 (take, then release). The host's desktop.trace.ndjson has no new PreviewManager.handleHumanInput spans during the test; the only ones are from using the tab directly on the host earlier.

    Path read from the 0.1.4 nightly bundle: viewer message → viewer.canOperate check → tab.control.human(viewer.id, …) → dispatchViewerInput → Input.dispatchMouseEvent on the tab's CDP session. For desktop-rendered tabs (tab.desktop), that session comes from connectDesktopPage. From the traces we can't tell whether the input is dropped on the server side or at the desktop endpoint.

    Also seen: the stream is blurry and letterboxed, because the viewer resize message returns early when tab.desktop is set.

  12. NdeFrutos commented on Oct 30, 2026

    @NdeFrutos
    NdeFrutos
    on Oct 30, 2026

    Confirmed again on a Windows desktop host running 0.1.6-nightly.20261030.3904, installed build commit 36dac688a17c (2026-10-30), with T3 Code desktop on a second computer as the viewer.

    The page is visible and "You have control" is shown, but the user cannot navigate Wikipedia by clicking or type into the page. The same tab works through the host's native desktop UI. Host preview_status reports human control but has no viewport field.

    We reproduced the missing-size coordinate gate using the exact installed server and web-viewer functions and the recorded status. Detailed evidence and verification limits are in #19456 (comment).

    This confirms the defect on this current Windows nightly; we have not tested the proposed patch or inspected the remote client's bundle.

    Posted by Codex on behalf of the account owner, with their authorization.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    via-triageFiled through npx t3 triage

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions