Repository navigation
Validate native text insertion on hosted channels and under interruption #65
Description
Status (Oct 6): implementation is merged (#81, #67). Remaining: hosted workspace execution of the new insertion route and broader refusal/interruption acceptance below. Clipboard ownership lifecycle is #78; public clipboard tools are githubnext/desktop-tools#3 (draft #75 was closed as superseded).
Extend the first native-action tools from #63 with contextual text selection, insertion at the caret, and keyboard shortcuts. Tracked by #8; pointer gestures and public clipboard tools have separate PRs.
- Select literal text or position the caret before/after it, using adjacent context to distinguish repeated matches and refusing ambiguous selections. Merged in Add native text selection and keyboard shortcuts #81.
- Navigation, letters/digits/function keys, and Command/Control/Option/Shift modifiers as complete press/release chords. Merged in Add native text selection and keyboard shortcuts #81, including the AXEnabled metadata fix (Do not reject editable text areas when AXEnabled is absent #66).
- Insert Unicode and multiline text at the observed caret/selection without submitting newlines or changing surrounding text. Merged in Add literal desktop insertion with reserved clipboard ownership #67, including the actual Ace WKWebView defect Preserve literal newlines when inserting text into web composers #76 and plain-text line-break fidelity.
- Preserve exact process/window/focused-control binding, single-use observations, native coordination, interruption evidence, and completed/refused/unknown outcomes. Mutations remain unsafe to replay; no global-input fallback.
- Temporary paste must verify meaningful consumption before restoring prior clipboard contents, preserve newer human copies, and retain ownership while delivery is uncertain (Keep clipboard ownership while a native paste is unresolved #78). Public clipboard tools are tracked in Add temporary clipboard paste and lifecycle validation desktop-tools#3.
- Access follows [Meta] Complete native computer use in Ace #8 and the current architecture: the channel owner's agent-access and desktop-tools settings, with no participant grants or per-action approvals.
Implementation status
Merged #81: contextual UTF-16 selection, caret movement, ambiguous/disabled refusal, complete shortcuts, deletion, and consumed-snapshot checks. Real AppKit and local/hosted pi checks exercised the unchanged selection/shortcut paths; persisted results replayed without repeating input. #80 separately repairs application activity/visibility metadata.
Merged #67: GUI-owned literal paste landed as 8fb8ccf from signed head eff5e49. Actual Ace composer validation proved contextual Unicode/multiline replacement, exact surrounding text and UTF-16 caret, observed consumption, restored clipboard, released ownership, and unchanged durable history across all chats. Plain-text paste/copy preserves single, blank, and edge line breaks. Two distinct signed clients verified delayed-paste ownership, refusal without input or clipboard change, and a fresh second insertion after read-only resolution. Real pi/Anthropic completed and Stop-during-pending-paste checks preserved exact results/images and usage after channel-worker restart, without repeating input. Stop correctly retained its partial-input warning; the original paste arrived once afterward. Type/lint checks, signed build/signatures, source review, and React Doctor passed.
Remaining validation: keep this issue open for the new literal-paste route’s hosted workspace execution and broader refusal/interruption coverage. Earlier hosted checks exercised selection/shortcuts and do not certify the new insertion route. Native-client death, response loss, GUI crash/restart, and separate-channel ownership cases remain in #78. Public clipboard tools are tracked in githubnext/desktop-tools#3.
Done when the merged selection/shortcut workflow and the merged literal-insertion route pass real native and pi/model checks, including ambiguous/stale/focus refusal, Stop/restart without replay, and hosted workspace execution. Unsupported delivery routes must remain explicit refusals.
Activity
iamnbutler commentedon Oct 5, 2026
Implementation and signed build are complete; real native validation found #66. Its fix uses Peekaboo’s knownIsEnabled property so unknown AXEnabled can proceed to native action validation while explicitly disabled controls remain refused. The signed rebuild passes; native/model/recovery checks are continuing. Issue #8 now includes the current validation status.
[-]Select native text, insert at the caret, and use keyboard shortcuts[/-][+]Validate native text insertion on hosted channels and under interruption[/+]iamnbutler commentedon Oct 13, 2026
Selection, shortcuts, and literal insertion shipped in #81 and #67. The native engine now lives in githubnext/desktop-tools (#213); remaining hosted/interruption validation belongs there. Closing in the pre-Universe cleanup.