Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions openspec/changes/add-speech-recognition/design.md
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,8 @@

## Validation

麦克风右键必须通过已有 `DesktopContextMenu` 委托协议接管捕获阶段的事件,按按钮实例确认归属;不能仅依赖按钮冒泡阶段的 `onContextMenu`。打开设备菜单时关闭已有通用菜单及延迟打开任务,编辑器区域继续使用复制/粘贴等通用动作。浏览器回归必须同时挂载真实全局菜单,验证按钮图标、空白区域和菜单切换。

配置解析/持久化、HTTP 参数映射与错误解析的 C++ 测试;录音状态机、WAV、配置同步的 JS 测试;pnpm test/build;Windows 定向构建和后端 HTTP 实测;浏览器检查主题、入口可见性、录音/停止/取消/发送及草稿隔离。最终审查原有脏改动仍保留。

### 当前验收证据(2026-10-18)
Expand Down Expand Up @@ -59,3 +61,10 @@
- 普通/首页、明暗主题和 390 px 窄屏的录音前后 x/y/width/height 差均小于 1 CSS px,菜单不含语言项。原工具栏持续占位,录音层覆盖显示。
- 最终 pnpm test、pnpm build、完整浏览器交互、OpenSpec 严格校验及 diff 检查通过。浏览器脚本通过真实按钮打开快捷键设置,避免测试驱动直接插入设置页时破坏聚焦编辑器的提交阶段;异步回填也不会再抢走弹窗焦点。
- 这些增补已更新源码和 Web 构建;先前 Windows 验证二进制未包含本轮增补,当前未替换运行中的 Desktop,也未追加 Mac/Linux 真机验收。

### 全局右键菜单回归修正(2026-10-19)

- 前轮浏览器夹具漏挂 `DesktopContextMenu`,只验证到按钮自身事件。补挂后准确复现右键麦克风出现“全选”:全局 document 捕获处理先于 React 按钮冒泡处理并阻止传播。
- 麦克风通过现有委托事件按按钮实例认领;打开时关闭通用菜单和待打开任务,保留无全局组件时的按钮处理。
- 带真实全局菜单的浏览器回归通过:图标和按钮空白区、明暗主题、已有编辑菜单切换、无 pointerdown 的 contextmenu 事件、编辑器粘贴/全选;原设备选择、快捷键编辑、按住/松开、迟到授权、布局稳定和录音完整流程仍通过。
- `pnpm test`、`pnpm build`、OpenSpec 严格校验及差异检查通过。本轮验证使用 Chromium 模拟音频设备和受控识别 API;未重新构建或替换正在运行的 Desktop。
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,10 @@
- **WHEN** 用户右键麦克风并选择一个音频输入设备
- **THEN** 下次录音使用该设备,菜单显示选中状态,设备不可用时提示重新选择且保留草稿

#### Scenario: 全局右键菜单共存
- **WHEN** 全局右键菜单已挂载,用户右键麦克风图标或按钮空白区域
- **THEN** 只打开该输入框的麦克风配置,关闭已有通用菜单,不显示“全选”;编辑器内右键仍显示原有编辑菜单

#### Scenario: 可配置快捷键
- **WHEN** 用户按 Ctrl+D 或在快捷键设置中自定义的录音快捷键
- **THEN** 在可用输入框开始录音,再按停止并回填;快捷键录入、弹窗及其他编辑区域不会误触录音
Expand Down
4 changes: 4 additions & 0 deletions openspec/changes/add-speech-recognition/tasks.md
Original file line number Diff line number Diff line change
Expand Up @@ -23,3 +23,7 @@
- [x] 6.1 麦克风 hover 提示“右键配置麦克风”;右键菜单列出实际音频输入设备、选中标记和按住录音开关,不含语言项,设备偏好在本机保存并用于实际采集。
- [x] 6.2 默认 Ctrl+D 开始/停止录音并接入快捷键设置、冲突校验及上下文守卫;按住模式松开即停止,迟到授权、窗口失焦和切换会话不残留录音。
- [x] 6.3 录音工具栏覆盖原工具栏且保持原布局尺寸;验证普通/首页、明暗、窄屏切换时边界不跳动及菜单、快捷键和录音完整流程。

## 7. 麦克风右键事件修正
- [x] 7.1 在浏览器回归中挂载生产用 DesktopContextMenu,复现右键麦克风误开“全选”;接入已有委托机制,使图标及按钮空白区均打开麦克风配置,保留编辑器通用菜单。
- [x] 7.2 验证菜单切换互斥、设备选择、快捷键和原有录音流程,运行 Web 测试/构建及 OpenSpec 严格校验,并记录实际验证范围。
37 changes: 35 additions & 2 deletions web/scripts/test-speech-recognition.mjs
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ import React,{useState,useRef} from 'react';
import {createRoot} from 'react-dom/client';
import '/src/styles/globals.css';
import {InputBar} from '/src/components/InputBar.jsx';
import {DesktopContextMenu} from '/src/components/DesktopContextMenu.jsx';
import {SettingsPage} from '/src/components/SettingsPage.jsx';
import '/src/i18n/index.js';
import {Toaster} from '/src/components/Toast.jsx';
Expand Down Expand Up @@ -72,7 +73,7 @@ function App(){
onSubmit={text=>window.fixture.sent.push(text)}
sessionControls={{model:'ACEModel-gpt-6-astra',modelOptions:[model],selectedModelName:model.name,
onModelChange(){},onRefreshModels(){},permissionMode:'default',reasoningOptions:{label:'极高',items:[]}}}/>
</div><button type="button" onClick={()=>{setSettingsNav('shortcuts');setSettingsOpen(true);}}>打开快捷键设置</button><Toaster/>
</div><button type="button" onClick={()=>{setSettingsNav('shortcuts');setSettingsOpen(true);}}>打开快捷键设置</button><Toaster/><DesktopContextMenu/>
</main>;
}
createRoot(document.getElementById('root')).render(<App/>);
Expand Down Expand Up @@ -197,6 +198,39 @@ try {
await page.waitForFunction(() => window.fixture.saved().base_url==='http://speech.internal:7777');
await settings.getByRole('button', { name: '收起', exact: true }).click();
await page.evaluate(() => window.fixture.settings(false));
const microphoneMenu=page.getByRole('menu',{name:'麦克风配置'});
const genericMenu=page.locator('.ace-desktop-context-menu');
// Include the document capture listener used in production, not only the button's React handler.
await page.getByRole('button',{name:'听写',exact:true}).click({button:'right'});
await page.waitForFunction(()=>document.querySelector('[aria-label="麦克风配置"],.ace-desktop-context-menu'));
assert.equal(await genericMenu.count(),0,'microphone right-click must not open the global Select All menu');
await microphoneMenu.waitFor();
await page.keyboard.press('Escape');
await editor.click({button:'right'});
await genericMenu.getByRole('menuitem',{name:'粘贴',exact:true}).waitFor();
assert.equal(await genericMenu.getByRole('menuitem',{name:'全选',exact:true}).count(),1,'editor retains the generic editing menu');
assert.equal(await microphoneMenu.count(),0);
// A context-menu event can arrive without pointerdown first (keyboard/native delegation).
await page.locator('[data-composer-speech] svg').dispatchEvent('contextmenu',{bubbles:true,cancelable:true});
await microphoneMenu.waitFor();
assert.equal(await genericMenu.count(),0,'opening microphone settings closes an existing generic menu');
await page.keyboard.press('Escape');
for (const theme of ['light','dark']) {
await page.evaluate(theme=>{document.documentElement.dataset.theme=theme;},theme);
await page.locator('[data-composer-speech] svg').click({button:'right'});
await microphoneMenu.locator('[data-microphone-device]').first().waitFor();
assert.equal(await genericMenu.count(),0,'right-clicking the icon reaches the microphone delegate');
await microphoneMenu.screenshot({path:path.join(output,`microphone-context-${theme}.png`)});
await page.keyboard.press('Escape');
await page.locator('[data-composer-speech]').click({button:'right',position:{x:2,y:14}});
await microphoneMenu.waitFor();
assert.equal(await genericMenu.count(),0,'button padding also belongs to the microphone menu');
await editor.click({button:'right'});
await genericMenu.getByRole('menuitem',{name:'粘贴',exact:true}).waitFor();
assert.equal(await microphoneMenu.count(),0,'editor context menu dismisses the microphone menu');
await page.keyboard.press('Escape');
}
console.log('[pass] real global context-menu delegation: icon, button padding, menu switching and editor actions');
const start = async () => {
const before=await page.locator('[data-speech-composer]').boundingBox();
await editor.click();
Expand Down Expand Up @@ -281,7 +315,6 @@ try {
const microphone=page.getByRole('button',{name:'听写',exact:true});
assert.equal(await microphone.getAttribute('title'),'右键配置麦克风');
await microphone.click({button:'right'});
const microphoneMenu=page.getByRole('menu',{name:'麦克风配置'});
await microphoneMenu.waitFor();
await microphoneMenu.locator('[data-microphone-device]').first().waitFor();
assert.equal(await microphoneMenu.getByText('Language',{exact:true}).count(),0);
Expand Down
26 changes: 24 additions & 2 deletions web/src/components/SpeechMicrophoneButton.jsx
Original file line number Diff line number Diff line change
@@ -1,9 +1,14 @@
import { useEffect, useRef, useState } from 'react';
import { useCallback, useEffect, useRef, useState } from 'react';
import { AnchoredMenu } from './AnchoredMenu.jsx';
import { Toggle } from './Modal.jsx';
import { VsIcon } from './Icon.jsx';
import { listSpeechDevices } from '../lib/speechDevices.js';
import { speechErrorMessage } from '../lib/speechRecognition.js';
import {
closeDesktopContextMenu,
CONTEXT_MENU_DELEGATE_EVENT,
SPEECH_MICROPHONE_CONTEXT_MENU_DELEGATE,
} from '../lib/desktopContextMenu.js';

function MicrophoneMenu({ speech, anchorRef, onClose }) {
const [devices, setDevices] = useState([]);
Expand Down Expand Up @@ -53,12 +58,29 @@ export function SpeechMicrophoneButton({ speech, disabled, onBeforeStart }) {
const anchor = useRef(null);
const [open, setOpen] = useState(false);
useEffect(() => { if (disabled || speech.active || !speech.available) setOpen(false); }, [disabled, speech.active, speech.available]);
const openMenu = useCallback(() => {
closeDesktopContextMenu();
if (!disabled && !speech.active && speech.available) setOpen(true);
}, [disabled, speech.active, speech.available]);
useEffect(() => {
const handler = (event) => {
const detail = event.detail;
if (!detail || detail.handled || detail.name !== SPEECH_MICROPHONE_CONTEXT_MENU_DELEGATE
|| detail.element !== anchor.current) return;
// The global menu owns document capture; claim only this composer's button.
detail.handled = true;
openMenu();
};
window.addEventListener(CONTEXT_MENU_DELEGATE_EVENT, handler);
return () => window.removeEventListener(CONTEXT_MENU_DELEGATE_EVENT, handler);
}, [openMenu]);
const beginHold = (kind) => { onBeforeStart?.(); speech.beginHold(kind); };
return <>
<button ref={anchor} type="button" data-composer-speech="true" aria-label="听写" title="右键配置麦克风"
data-desktop-context-menu-delegate={SPEECH_MICROPHONE_CONTEXT_MENU_DELEGATE}
aria-haspopup="menu" aria-expanded={open} disabled={disabled}
className="w-7 h-7 rounded-full shrink-0 flex items-center justify-center text-fg-mute hover:bg-surface-hi hover:text-fg disabled:opacity-50 focus-visible:outline-2 focus-visible:outline-accent"
onContextMenu={(event) => { event.preventDefault(); event.stopPropagation(); if (!disabled) setOpen(true); }}
onContextMenu={(event) => { event.preventDefault(); event.stopPropagation(); openMenu(); }}
onPointerDown={(event) => {
if (event.button === 0 && speech.preferences.holdToRecord) { event.preventDefault(); beginHold({ kind: 'pointer', id: event.pointerId }); }
}}
Expand Down
1 change: 1 addition & 0 deletions web/src/lib/desktopContextMenu.js
Original file line number Diff line number Diff line change
Expand Up @@ -99,6 +99,7 @@ export const CLOSE_DESKTOP_CONTEXT_MENU_EVENT = 'acecode:close-desktop-context-m
export const CONTEXT_MENU_DELEGATE_EVENT = 'acecode:context-menu-delegate';
export const CONTEXT_MENU_DELEGATE_SELECTOR = '[data-desktop-context-menu-delegate]';
export const SESSION_HEADER_CONTEXT_MENU_DELEGATE = 'session-header';
export const SPEECH_MICROPHONE_CONTEXT_MENU_DELEGATE = 'speech-microphone';

// Explicit toolbar entry point; native right-click continues through contextmenu.
export function openDesktopContextMenu(options) {
Expand Down
Loading