diff --git a/.github/workflows/quality.yml b/.github/workflows/quality.yml index 4d0c0814..f03f8051 100644 --- a/.github/workflows/quality.yml +++ b/.github/workflows/quality.yml @@ -120,9 +120,20 @@ jobs: # snapshot of `je_auto_control/` straight into site-packages and masks # the editable install for any sub-package the snapshot doesn't include # (admin, usb, remote_desktop, vision, …). - - name: Install the project itself + # + # `[webrtc]` is part of the install and is load-bearing for what this job + # measures. Eleven modules under `utils/remote_desktop` raise ImportError + # at module level without `aiortc`/`av` — 2,090 statements that were a + # hard 0% here no matter what anyone wrote. Worse than the number: the + # tests that cover the WebRTC host's auth, TLS, tokens and file transfer + # were already written and `importorskip`ped straight past on every + # square, so they ran on developer machines and nowhere else. Measured on + # this tree, one variable changed: 513 of those statements are covered by + # tests that exist today. The extra is NOT added to `typing-stable-api` + # below — that gate must not depend on what is installed. + - name: Install the project itself, with the WebRTC extra shell: bash - run: pip install -e . # NOSONAR githubactions:S8544 githubactions:S8541 # reason: installs the checked-out project itself, so there is no upstream version to lock and no third-party setup script to run + run: pip install -e ".[webrtc]" # NOSONAR githubactions:S8544 githubactions:S8541 # reason: installs the checked-out project itself, so there is no upstream version to lock and no third-party setup script to run - name: Install the test tooling shell: bash diff --git a/CHANGELOG.md b/CHANGELOG.md index 18911307..e22a5e06 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,6 +10,13 @@ only when documented here with a migration path. ### Added +- **`ac_rrule_next`, `ac_rrule_occurrences` and `ac_format_date` now declare + the string format they parse.** Their `dtstart` / `now` / `value` properties + carry `"format": "date-time"` (or `"date"`) in the tool's input schema, which + the descriptions already said in prose and the schema did not. A client + generating values from the schema alone used to produce a plain string and + get a `ValueError` out of `datetime.fromisoformat`. + - **Windows on arm64 installs.** `opencv-python`, `cryptography` and `je_open_cv` now carry the environment marker `sys_platform != 'win32' or platform_machine != 'ARM64'`, because none of @@ -380,6 +387,47 @@ only when documented here with a migration path. ### Fixed +- **Changing a hotkey's combo on X11 left the old key grabbed for the life of + the daemon.** `LinuxHotkeyBackend._sync_one` dropped the previous + registration from its own table without calling `ungrab_key`, so the *old* + combo stayed grabbed on the X server: it was swallowed from every + application, fired nothing, and `_ungrab_all` could not release it at + shutdown because it no longer knew about it. Rebinding `ctrl+alt+k` to + something else made `ctrl+alt+k` dead system-wide until the process exited. + The Windows backend has always unregistered at the same point; the X11 one + now does too. Unaffected on Windows and macOS. + +- **A window closing mid-call let a COM error escape every Windows + accessibility read.** `comtypes` reports a provider failure as `COMError`, + which derives straight from `Exception` — the reason + `windows_query._uia_errors()` exists — but only the two tree-walking guards + in `backends/windows_backend.py` used that tuple. The other 37, covering + every control pattern (`get_value`, `invoke`, `toggle`, `read_table`, the + text and grid reads, …), named `(OSError, AttributeError, …)` and therefore + contained none of them. An application that stopped responding, or a window + that closed between the search that found an element and the call that read + it, raised `COMError` out of the `ac_*` tool or `AC_*` command instead of + answering `None` / `False` / `[]`, and past the executor's + `AutoControlException` boundary. All 37 now use the same tuple. This only + widens what is caught: no call that used to succeed behaves differently. + +- **The WebRTC viewer ended every clean disconnect with an unhandled task + exception.** `WebRTCDesktopViewer._consume_video` caught + `(OSError, RuntimeError)`, but aiortc signals the end of a track by raising + `MediaStreamError`, which derives straight from `Exception` and so matched + neither. Nothing awaits that task, so the normal end of a session — the host + stopping its screen share, or the connection closing — reached the console as + asyncio's "Task exception was never retrieved" traceback instead of the + "video stream ended" line the host's own drain loop already logged. The + stream is unaffected either way; only the logging changes. + +- **A `null` in a remote-desktop entry's `tags` became a tag named `"None"`.** + `AddressBook.set_tags()` cleaned its input with `str(t).strip()`, and + `str(None)` is the non-empty string `"None"`, so a JSON `null` in the array — + what a client sends for an omitted tag — was stored as a tag and then listed + by `all_tags()` alongside the real ones. Nulls are now dropped. Tags that + were already stored this way stay until the entry's tags are set again. + - **Typing text through the key-event route raised `AttributeError` on the three platforms that cannot do it.** `type_unicode_keys()` (and `AC_type_unicode_keys` / `ac_type_unicode_keys`) called the backend's diff --git a/CLAUDE.md b/CLAUDE.md index d8e97752..5815281e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -24,7 +24,7 @@ Layering: entry points (`cli.py`, `gui/`, socket / REST / MCP servers) → execu ```bash pip install -r dev_requirements.txt # dev deps -pip install -e .[gui] # + GUI extra +pip install -e .[gui,webrtc] # + GUI and WebRTC extras python -m pytest test/unit_test/headless # headless unit tests python -m pytest test/integrated_test/ # cross-module workflows python -m coverage run -m pytest # the suite WITH coverage (see below) @@ -41,6 +41,13 @@ have their import-time lines recorded as never executed: measured, that is suite). `test/unit_test/headless/test_coverage_measurement.py` holds CI to the correct spelling. +**Measure it with the `[webrtc]` extra installed**, which is why it is in the +line above. Eleven modules under `utils/remote_desktop` raise `ImportError` at +module level without `aiortc`/`av` — 2,090 statements, about 4 points — and the +tests covering the WebRTC host's auth, TLS, tokens and file transfer +`importorskip` straight past. `quality.yml` installs the extra so the floor is +measured against the same tree a developer sees. + `pyproject.toml` pins `python_files = ["test_*.py"]` on purpose: the `*_test.py` files under `test/unit_test/` are manual demo scripts whose module bodies drive the real mouse and keyboard on import. Never loosen that setting. ## Feature Delivery Rules diff --git a/Progress.md b/Progress.md index 153e4c4d..82b2a8c1 100644 --- a/Progress.md +++ b/Progress.md @@ -25,7 +25,7 @@ | --- | ---: | --- | | `utils/mcp_server/tools/_handlers.py` | 4,789 | 676 個 MCP 工具的處理函式本體。與 `_factories.py`（表）不同,這裡是邏輯,應該依主題拆成 `_handlers/` 套件（input／screen／window／file／agent…）。拆點清楚,純粹是量大。 | | `gui/remote_desktop/webrtc_panel.py` | 2,545 | 單一 Qt 面板,但已含連線、監視器選擇、頻寬自適應、麥克風、錄影五組互動狀態。應拆成 panel + 各控制器。 | -| `utils/accessibility/backends/windows_backend.py` | 915 | 已拆出 `windows_query.py`（170）與 `windows_state.py`（98）。剩下的是同一套 UIA COM 生命週期管理,再拆會把 `CoInitialize`／介面釋放的配對邏輯切散。 | +| `utils/accessibility/backends/windows_backend.py` | 923 | 已拆出 `windows_query.py`（170）與 `windows_state.py`（98）。剩下的是同一套 UIA COM 生命週期管理,再拆會把 `CoInitialize`／介面釋放的配對邏輯切散。**2026-08-24 從 918 長到 923**:見下面的說明。 | **本質豁免（依 `CLAUDE.md` 的「flat data tables」條款,不算既有豁免）**: `utils/mcp_server/tools/_factories.py`（8,972,MCP 工具註冊表）、 @@ -49,6 +49,17 @@ 行數沒有任何 CI 在把關（`quality.yml` 只跑 ruff 與 bandit,而 ruff 只管行寬), 所以這張表只會在有人手動實測時才會被發現對不上——上次就是。 +### 2026-08-24:`windows_backend.py` 從 918 長到 923，理由記在這裡 + +表上原本寫 915，2026-08-24 實測時工作樹已經是 **918**（表本身就過期了， +正是上一段講的那件事）。這次又 +5，是為了改掉一個真的錯：檔內 37 個 +`except (OSError, AttributeError, …)` 攔不到 comtypes 的 `COMError` +（細節見下面覆蓋率那一節與 [CHANGELOG.md](CHANGELOG.md)）。5 行是一個模組層 +常數加兩行註解——`CLAUDE.md` 允許「超標檔案再變長」的兩條路是**先拆**或 +**在這裡寫明為什麼不拆**，這是後者：拆這個檔的正確切點是 UIA COM 的生命週期 +管理，和這次的修正無關，綁在一起會讓一個三行的正確性修補變成大面積 diff。 +**新上限是 923**，規則不變。 + --- ## Windows arm64:裝得起來了，但少了影像與加密 @@ -220,22 +231,32 @@ capability enum 值與 variadic `ei_seat_bind_capabilities`、event-type enum --- -## 兩個門檻：mypy 那半到終點了，覆蓋率那半是量錯了 +## 兩個門檻：mypy 那半到終點了，覆蓋率那半在往 80 爬 -`DECIDE` — 兩半都做完了，只剩「下一個覆蓋率目標是多少」要維護者拍板 +`TODO` — 型別那半 2026-08-22 收工；覆蓋率那半有目標了（80），還沒到 原本這一條記的是兩個只存在於 `pyproject.toml` 註解裡、沒有任何機制的承諾。 -2026-08-21 把**機制**補上了（做法見 [WHATS_NEW.md](WHATS_NEW.md)），兩半也都走完了： +2026-08-21 把**機制**補上了（做法見 [WHATS_NEW.md](WHATS_NEW.md)）： 型別契約的豁免清單 2026-08-22 清空，平台縫最後兩個名稱（`keyboard`／`mouse`） 2026-08-23 拿到合約；覆蓋率那半發現不是爬得不夠，是量測起點錯了，修正後地板 -從 50 提到 69。 +從 50 提到 69，隔日再提到 75。 + +**2026-08-23 維護者拍板：下一個覆蓋率目標是 80。** 這一條留到那時候。 +型別那一節留著是因為它記的那幾個坑之後還會踩到。 + +### 覆蓋率：目標 80 達成，地板提到 81——**這一條結束了** -**這一條還留著，是因為只剩一個問題要維護者回答：下一個覆蓋率目標值是多少。** -另外兩節留著是因為它們記的那幾個坑之後還會踩到。 +`TODO` → **完成（2026-08-24）**。2026-08-23 拍板的目標是 **80**；九宮格實測 +最低那一格 **81.40%**（ubuntu-22.04／3.14），最高 82.73%（windows-2022／3.12）， +`fail_under` 隨之從 75 提到 **81**。 -### 覆蓋率：目標 70 其實早就到了，是量錯了 +從 75 到 81 是五批：WebRTC 那一族、window backends、accessibility backends、 +hotkey backends，以及最後這一批——掃不到的那批 adapter（下面第 2 項那條 +`DECIDE`，2026-08-24 拍板走「建真的實例」）。 -`DECIDE` — 地板已經設成修正後矩陣的最低那一格（69）；**下一個目標值要維護者定** +**這一整節之後只剩參考價值**：它記的幾個坑（量測起點、地板只能從 +`coverage report` 讀、只補跑得到九格的程式碼）之後每次動覆蓋率都還會踩到， +所以留著。下一個目標由維護者再定。 `fail_under` 一度從 35 提到 50，理由寫在 `pyproject.toml`。**那兩個數字都低了大約 24 點**，而原因不在測試，在量測的起點： @@ -270,19 +291,293 @@ import 期程式碼（`def` 行、類別本體、常數、兩張大分派表） 地板因此設成 **69**——取最低那一格往下取整，與當初 50 取自 50.26% 是同一個慣例。 `[tool.coverage.report]` 的 `precision` 也從預設的 0 提到 2：預設精度下九格全部印 -「70%」，而它們其實是 69.67 到 70.97，害得這次的地板得去 XML artifact 裡撈。 +「70%」，而它們其實是 69.67 到 70.97，分不出最低的是哪一格。 順帶把 `fail_under` 的容差從一整個百分點縮到 0.01。 地板只有一個家（`pyproject.toml` 的 `fail_under`），`quality.yml` 不再另外抄一份。 +**這一段原本寫「地板得去 XML artifact 裡撈」，那是錯的，2026-08-23 實測更正。** +上面那九個數字是 `coverage report` 印的，也就是 `fail_under` 真正比對的那個數字， +而它**含分支**（`[tool.coverage.run]` 的 `branch = true`）；`coverage.xml` 的 +`line-rate` 屬性**不含分支**，所以同一格會高出約 1.8 點——a585e65 那一格 +report 印 69.67%，artifact 是 71.07%。照 artifact 設地板，設出來的會是這套測試 +過不了的地板。要看單一子系統的缺口用 artifact，要設地板只能用 report。 + Windows 是高的那一角，因為門面 import 進來的是**它自己那個平台的後端**； 換句話說剩下的那 30 點裡，有一部分是任何單一平台都拿不到的。 -**還要決定的**：70 是舊的目的地，現在等於已經到了，**下一個目標值該由維護者定**。 -真正還低的是哪幾塊，現在有實測（本機 Windows／3.14，修正後）： -`utils/remote_desktop` 35%、`utils/mcp_server` 34%（`_handlers.py` 自己 10%）、 -`utils/executor` 41%、`utils/accessibility` 29%、`wrapper/window_backends` 10%。 -這四塊的共同形狀是「一大堆薄轉接函式包著已經測過的無頭函式」，所以往上爬的方式 -是走註冊表逐一驅動，而不是一支一支手寫測試。 +#### 往 80 怎麼爬：走註冊表，而且替身要從型別合約長出來 + +第一批 2026-08-23 落地（`test/unit_test/headless/test_adapter_registry_sweep.py`， +做法見 [WHATS_NEW.md](WHATS_NEW.md)）。兩個註冊表、一支測試檔： + +| 掃什麼 | 參數從哪來 | 換掉什麼 | +| --- | --- | --- | +| 657 個 MCP 工具 | 工具自己宣告的 JSON schema（`_factories.py`） | 被呼叫者 | +| 773 個 `AC_*` 命令 | Script Builder 的 `command_schema.py`；沒有 spec 的用參數標注 | 被呼叫者 | + +**關鍵不是「掃」，是替身怎麼來的**：替身的回傳值是從**被呼叫者自己的回傳標注** +造出來的（`Optional[X]` → `None`、容器 → 空容器、純量 → 零值）。這件事 +2026-08-22 之前做不到——那時候還有 136 個模組不在型別契約裡，沒有東西可讀。 +契約清空之後，「這個轉接函式有權假設什麼」變成程式讀得出來的東西，於是 460 個 +MCP adapter 與 372 個執行器 adapter 可以在沒有滑鼠、沒有螢幕、沒有網路的情況下 +整段跑完，而且跑的是**剛好等於合約承諾的東西，不多也不少**。 + +實測（本機 Windows／3.14）： + +| | 之前 | 之後 | +| --- | ---: | ---: | +| `_handlers.py` | 37.47% | **75.21%** | +| `action_executor.py` | 55.78% | **73.52%** | +| 全專案 | 71.91% | **74.91%** | + +掃不到的是兩種形狀，而且是刻意的：adapter 伸手進**兩個**專案模組（那是組合， +不是轉接），以及 adapter 自己 `import` 第三方套件（那是它在挑後端，回傳取決於 +機器）。三個 MCP adapter 需要的比合約承諾的更多，用名字列在測試檔裡，各附一行 +理由；**其中任何一個哪天開始通過，測試會要求把它刪掉**，清單不會爛在那裡。 + +#### 2026-08-23 拍板：`quality.yml` 裝 `[webrtc]` extra + +原本這裡是一條 `DECIDE`，寫的是「要嘛讓 CI 裝那個 extra，要嘛承認那 2,900 個 +statement 是分母裡的死重」。**維護者選了裝**，理由不是那一點覆蓋率： + +`utils/remote_desktop` 底下有 **11 個模組在沒有 `aiortc`／`av` 時於模組層拋 +ImportError**，共 2,090 個 statement——在 CI 上是硬性的 0%，寫什麼測試都動不了。 +比數字嚴重的是另一件事：**涵蓋 WebRTC host 的 auth、TLS、resume token、檔案傳輸的 +測試早就寫好了**，只是每一格都 `importorskip` 略過，等於只在開發機上跑過。 + +只換這一個變數實測（同一台機器、同一套測試）：那 2,090 個裡有 **513 個**是現有測試 +就會蓋到的，端到端 +1.23 點（windows-2022／3.14）。相依在九宮格上都解得開 +（`aiortc` 1.15.0 + `av` 17.1.0 對 win_amd64／manylinux x86_64／macos-14 arm64 的 +3.10 與 3.14 都有 wheel）。`typing-stable-api` **刻意不裝**——那個閘門的判定不能隨 +環境浮動，理由在下一節。 + +`dev_requirements.txt` 與 `CLAUDE.md` 的開發指令也跟著加了那個 extra：少裝它的人 +量到的數字會比 CI 執行的地板低約 4 點。 + +#### 三個註冊表都掃過了，替身再往合約深一層 + +`test_adapter_registry_sweep.py` 之後又走了三步（做法見 [WHATS_NEW.md](WHATS_NEW.md)）： + +| 這次多掃到的 | 為什麼原本掃不到 | +| --- | --- | +| 92 個 adapter：被呼叫者回傳專案 dataclass | `_value_for` 只認純量與容器，dataclass 直接被判成「模不出來」。現在從 dataclass **自己的欄位標注**造實例，於是 adapter 的 `.to_dict()` 也一起跑起來 | +| 101 個 adapter：被呼叫者是模組層單例的方法 | 匯入的名字不是 function 而是 `default_observer`／`default_scheduler`／`registry` 這種物件。呼叫**單一**方法就是同一個轉接形狀往下一層，方法的標注就是合約；呼叫兩個以上算編排，仍然排除 | +| 31 條 REST 路由 | `rest_handlers` 是同一形狀的第三個註冊表，而現有的 REST 測試走 HTTP 層，在無頭 runner 上多半只是看著 handler 掉進自己的 `except` 回 500 | + +REST 那一支的參數來自 `rest_openapi.build_openapi_spec()`——與 handler 不同檔， +所以掃不出「拿被測程式當答案」的循環；順帶白拿一條契約測試：**路由表與 OpenAPI +文件必須描述同一組 API**，多一條少一條都當場紅。 + +三支共用的機器搬進 `test/unit_test/headless/_contract_sweep.py`，各自只留自己的 +參數來源。 + +#### 九宮格怎麼走到 81 的（每一列都是 `coverage report` 實測） + +| | 最低 | 最高 | 地板 | +| --- | --- | --- | ---: | +| PR #486 首輪 | 69.67%（ubuntu-22.04／3.14） | 70.97%（windows-2022／3.12） | 69 | +| 補完 WebRTC 之後 | 75.79%（ubuntu-22.04／3.14） | 76.99%（windows-2022，3.11／3.12） | 75 | +| 補完後四批之後 | **81.40%**（ubuntu-22.04／3.14） | 82.73%（windows-2022／3.12） | **81** | + +這九個數字讀的是 `coverage report`（也就是 `fail_under` 真正比對的那個）， +不是 artifact 的 `line-rate`——理由見上一節。最低那一格始終是 Linux、最高那一格 +始終是 Windows，因為門面 import 進來的是它自己那個平台的後端；換句話說 +**剩下的那 18 點裡有一部分是任何單一平台都拿不到的**，要再往上只能補在九格 +都跑得到的程式碼上。 + +#### 剩下的 4 點在哪裡（2026-08-24，ubuntu-22.04／3.14 的 artifact） + +以下是 statement 數（artifact 的口徑，不含分支），拿來看缺口分佈： + +| 子系統 | 沒蓋到 / 總 statement | 覆蓋率 | +| --- | ---: | ---: | +| `utils/remote_desktop` | 2,297 / 6,622 | 65.3% | +| ~~`utils/accessibility`~~ | ~~824 / 1,397~~ | **2026-08-24 補完（backends 全數 99–100%）** | +| `utils/executor` | 653 / 3,906 | 83.3% | +| `utils/usb` | 573 / 2,137 | 73.2% | +| `utils/mcp_server` | 573 / 4,618 | 87.6% | +| ~~`wrapper/window_backends`~~ | ~~361 / 477~~ | **2026-08-24 補完（100%）** | +| ~~`utils/hotkey`~~ | ~~221 / 426~~ | **2026-08-24 backends 全數 100%** | +| `utils/rest_api` | 146 / 808 | 81.9% | + +**要動地板，只能補在每一格都跑得到的程式碼上。** 地板取的是最低那一格， +所以只在 Windows 跑得到的東西補再多也不會動它。把九格的未覆蓋行取交集， +2026-08-24 量到 **9,343 個 statement 在每一格都沒被執行過**——那就是可攜的缺口， +也是唯一會抬地板的地方。最大的幾塊： + +| 檔案 | 每一格都沒蓋到 | 備註 | +| --- | ---: | --- | +| ~~`utils/executor/action_executor.py`~~ | ~~589~~ | **2026-08-24 補到 79.76%**（本機實測 77.31% → 79.76%，未覆蓋 589 → 498） | +| ~~`utils/accessibility/backends/windows_backend.py`~~ | ~~446~~ | **2026-08-24 補完（99.42%）**，順便修掉 37 個攔不到 `COMError` 的 except | +| ~~`utils/remote_desktop/webrtc_viewer.py`~~ | ~~354~~ | **2026-08-24 補完（100%）** | +| ~~`utils/remote_desktop/webrtc_host.py`~~ | ~~351~~ | **2026-08-24 補完（100%）** | +| ~~`utils/mcp_server/tools/_handlers.py`~~ | ~~348~~ | **2026-08-24 補到 86.73%**（本機實測 83.23% → 86.73%，未覆蓋 348 → 262） | +| `utils/remote_desktop/signaling_server.py` | 155 | **CI 動不了**：要 `[signaling]` extra（fastapi／uvicorn），沒裝 | +| ~~`utils/remote_desktop/multi_viewer.py`~~ | ~~146~~ | **2026-08-24 補完（100%）** | +| ~~`wrapper/window_backends/x11_backend.py`~~ | ~~133~~ | **2026-08-24 補完（100%）**。「只有 Linux 那兩格跑得到」是錯的，見下 | +| ~~`utils/accessibility/backends/linux_backend.py`~~ | ~~132~~ | **2026-08-24 補完（100%）** | + +**下一步的順序**： + +1. ~~`webrtc_host`／`webrtc_viewer`／`multi_viewer`／`webrtc_transport`／ + `webrtc_audio`~~ **2026-08-24 做完**，見下一節。 +2. ~~掃不到的那批 adapter：卡在「被呼叫者是 class」~~ **2026-08-24 做完**。 + 原本寫的兩個選項是個假二選一：第三條路是照 class 自己的 `__init__` 標注 + **建真的實例**，不用方法替身，所以「跑起來了」和「驗到了東西」不會分家。 + 維護者拍板走這條，見下下節。 +3. ~~`wrapper/window_backends` 與 `utils/accessibility`~~ + **2026-08-24 兩個都整包補完**，見下下節。 + +`utils/office`（77）與 `signaling_server`（155）**不要碰**：`quality.yml` 沒裝 +`[office]`／`[signaling]`，補的測試會整批 skip，對地板一個點都不動。要補之前 +先照 `[webrtc]` 的先例把 extra 加進 CI。 + +#### 2026-08-24：WebRTC 那一族補完了，而「先拆一半」是問錯了問題 + +上面第 1 項原本寫著「`webrtc_host` 與 `webrtc_viewer` 是兩個大類別， +**先看能不能拆出可測的那一半**」，理由是那兩個 mixin +（`webrtc_host_auth`／`webrtc_host_media`）是拆出來才測得到的先例。 +**實際去看之後發現不必拆**：兩個類別的建構子都不碰 aiortc——主機只存下 config、 +RateLimiter 與 SessionPermissions，檢視端只存下 callback——而每一個協作對象 +（`RTCPeerConnection`、`ScreenVideoTrack`、asyncio 橋接、稽核記錄）都是 +模組層名稱或建構參數，換掉就好。擋在 19.83%／14.08% 的從來不是類別的形狀， +是「要跑到第一行得先站起一個 PeerConnection、一個螢幕擷取器和一條背景事件迴圈」。 + +| 模組 | 之前 | 之後 | +| --- | ---: | ---: | +| `webrtc_host.py` | 19.83% | **100%** | +| `webrtc_viewer.py` | 14.08% | **100%** | +| `multi_viewer.py` | 21.24% | **100%** | +| `webrtc_audio.py` | 0.00% | **96.27%** | +| `webrtc_transport.py` | 27.89% | **88.84%** | + +沒補完的兩塊是刻意的：`webrtc_transport._get_cursor_position` 是三條平台分支， +任何一格只跑得到自己那條；`webrtc_audio._enqueue` 裡兩個吞掉的 queue 競態 +只有「謊報自己滿了的 queue」造得出來，那是在測替身不是在測程式。 + +替身集中在 `test/unit_test/headless/_webrtc_doubles.py`（形狀比照 +`_contract_sweep.py`），六個測試檔共用；`FakePeerConnection` 刻意把主機端與 +檢視端的介面放在同一個類別裡——它替的是同一個 aiortc 型別，照方向拆成兩個 +只會讓兩份替身各自漂走。 + +**要拆的是測試檔，不是被測的類別**，因為 `CLAUDE.md` 的 750 行上限對新檔案 +沒有例外。主機拆成 `test_webrtc_host_session.py`（557，offer／answer／狀態／拆除） +與 `test_webrtc_host_channels.py`（739，四條 DataChannel、權限、限流、收件匣）； +檢視端拆成 `test_webrtc_viewer_session.py`（466）／ +`test_webrtc_viewer_media.py`（385，m-line 位置與開關）／ +`test_webrtc_viewer_control.py`（519）。 + +**地板還沒動。** 本機（Windows／3.14）全專案 77.04% → 79.40%，363 個新測試， +但地板取的是九宮格最低那一格，**要等 CI 的 `coverage report` 印出來才能改**—— +這條規則在上面那節寫過，不從單機的數字推。 + +#### 2026-08-24：平台後端不是「只有那一格跑得到」，是沒人給過替身 + +上面第 3 項與缺口表都寫著 `x11_backend.py`／`linux_backend.py` +「只有 Linux 那兩格跑得到」。**實測是錯的**：`wrapper/window_backends/` 底下 +八個平台模組，每一句 `import Xlib`／`import Quartz`／`import AppKit`／ +`import ApplicationServices`／`import comtypes` **全部在函式內**，所以在一台 +沒裝任何一個的 Windows 上這八個模組都 import 得起來—— + +```bash +python -c "import je_auto_control.wrapper.window_backends.x11_backend" # 在 Windows 上就過 +``` + +擋住它們的從來不是平台，是**沒有替身**。把套件塞進 `sys.modules` 之後， +同一份測試在九格都跑得到，而不是只有兩格；地板取最低那一格，所以 +「九格都漲」比「兩格漲」更有意義。 + +| 檔案 | 之前 | 之後 | +| --- | ---: | ---: | +| `window_backends/x11_backend.py` | 0.00% | **100%** | +| `window_backends/macos_backend.py` | 0.00% | **100%** | +| `window_backends/__init__.py`（後端選擇） | 46.34% | **100%** | +| `window_backends/base.py` | 63.64% | **100%** | +| `window_backends/windows_backend.py` | 89.19% | **100%** | +| `window_backends/null_backend.py` | 100% | 100% | + +**替身要對得上真貨，否則只是自己跟自己同意。** 兩支 stub 的處理不一樣， +因為兩邊的常數性質不同： + +- `_xlib_stub.py` 的常數**帶真值**（`SubstructureRedirectMask` 就是 `1 << 20`）， + 因為那些數字會真的上線；`test_xlib_stub_values.py` 在有裝 python-Xlib 的地方 + （CI 的兩格 Linux）逐一比對，對不上就當場紅。本機另外用 + `pip install --target` 拉 0.33 實測過一輪，12 個常數全中。 +- `_pyobjc_stub.py` 的常數**是哨兵**，因為它們不是 dict key 就是原封不動傳回 + 同一支 stub 函式的 token，數值到不了任何算術；`test_pyobjc_stub_names.py` + 改成在 macOS 那兩格比對**名字存在**，另外只釘那三個真的會做位元運算的 + window-list 旗標。 + +**一個踩到的坑**：`from je_auto_control.windows.window import windows_window_manage` +這種 `from 套件 import 名字`，Python **先用套件的屬性解析**，解析不到才回頭查 +`sys.modules`。只把替身放進 `sys.modules["...windows_window_manage"]` 在 +Windows 上完全沒效果——真的 Win32 模組會被呼叫。要換掉的是**那個套件**。 + +本機（Windows／3.14）全專案 79.40% → **80.22%**，184 個新測試。 +地板一樣要等九宮格。 + +#### 2026-08-24：`utils/accessibility` 照抄同一招，並抓到 37 個攔不到的 except + +同一個事實在這裡也成立——三個平台後端的 comtypes／pyobjc／D-Bus import +全在函式內，所以塞 `sys.modules` 就能在九格都跑： + +| 模組 | 之前 | 之後 | +| --- | ---: | ---: | +| `backends/windows_backend.py` | 17.72% | **99.42%** | +| `backends/linux_backend.py` | 42.41% | **100%** | +| `backends/macos_backend.py` | 0.00% | **100%** | +| `backends/windows_query.py` | 24.75% | **99.01%** | +| `backends/windows_state.py` | 18.75% | **100%** | +| `backends/base.py` | 74.03% | **100%** | +| `backends/__init__.py` | 48.94% | **100%** | + +剩的三行在 `_process_name` 的 Win32 失敗路徑（`OpenProcess` 成功但 +`QueryFullProcessImageNameW` 失敗），要一個「開得到卻查不到」的行程才踩得到。 + +**AT-SPI 那一層要換的是 `SessionBus`，不是 `_AtspiConnection`。** 現有的 +`test_accessibility_linux.py` 換掉後者，那對測「走訪」是對的，但底下整個 +D-Bus 呼叫層一行都沒跑過——而協定就住在那裡（無障礙匯流排不是 session bus、 +accessible 是 `(sender, path)` **配對**、狀態位元是**兩個 32-bit word**， +只讀第一個會靜靜丟掉第 31 位以上的每一個狀態）。 + +**寫測試時抓到一個真的錯，形狀和 WebRTC 那個一模一樣：攔截 tuple 漏了型別。** +comtypes 把 provider 失敗報成 `COMError`，而它直接繼承 `Exception`： + +```python +issubclass(COMError, (OSError, AttributeError, ValueError, TypeError)) # False +``` + +`windows_query._uia_errors()` 存在就是為了講這件事，docstring 也點名了情境 +（「視窗在走訪途中關掉，或應用程式停止回應」）。但 `windows_backend.py` 裡 +只有**兩個**走訪用的 except 用了那個 tuple，**另外 37 個**——也就是每一個 +control pattern——寫的是 `(OSError, AttributeError, …)`，一個都攔不到。 +race 很小但真實：`_find_raw` 自己會攔，所以曝露的是「找到之後、讀之前」 +那一段。已全部改用同一個 tuple；這只會**放寬**攔截範圍。 + +#### 2026-08-24：`utils/hotkey/backends` 補完，**所有 backend seam 都有覆蓋了** + +`CLAUDE.md` 列的 backend seam 有八個（accessibility／ocr／vision／llm／agent／ +hotkey／usb／usbip）。ocr、vision、llm、agent 本來就只差個位數， +accessibility 與 window 這兩天補掉，剩下的就是 hotkey：三個平台、三套完全 +不同的機制（`RegisterHotKey` + 訊息幫浦／`XGrabKey`／`CGEventTap` + run loop）， +12.64%／24.46%／40.94% **全部到 100%**。 + +一樣都不需要桌面。Windows 那支把 `user32` 當**參數**傳給真正做事的三個方法， +所以錄音機式的替身在哪裡都能驅動；只有組出 `user32` 的開頭需要 +`ctypes.wintypes`，那兩支測試標成只在 Windows 跑。 + +**又抓到一個真的錯，這次是資源洩漏。** X11 的 `_sync_one` 在 combo 改掉時 +只把舊登記從自己的表裡拿掉，**沒有 `ungrab_key`**——舊的組合鍵於是一直被 +grab 在 X server 上，被所有應用程式吞掉、什麼也不觸發，而且 `_ungrab_all` +關機時也放不掉（它已經不知道那筆了）。使用者把 `ctrl+alt+k` 改成別的， +`ctrl+alt+k` 就在整個桌面上死到行程結束為止。 + +同一個檔案其實知道這個形狀——`_grab_masked` 的回滾註解就寫著「留著不放、 +`_registered` 又沒更新，會洩漏 grab 並在每次輪詢時噴 BadAccess」——而 Windows +那支一直都在同一個位置 unregister。漏的只有改綁定這條路。 + +X11 那批測試也把 Xlib stub 從 12 個常數長到 18 個外加一張 keysym 表， +全部照舊在 Linux 兩格對真貨比對（本機也用 python-Xlib 0.33 實測過：31 中 31）。 ### mypy：整包把關，**豁免清單已經清空** diff --git a/WHATS_NEW.md b/WHATS_NEW.md index dcc58e16..e2a0de2c 100644 --- a/WHATS_NEW.md +++ b/WHATS_NEW.md @@ -1,5 +1,511 @@ # What's New — AutoControl +## What's new (2026-08-24) + +### The Floor Is 81, And The Target Set On 2026-08-23 Is Met + +The nine-way matrix now runs **81.40%** (ubuntu-22.04 / 3.14) to 82.73% +(windows-2022 / 3.12), so `fail_under` goes from 75 to 81 — floored to the +integer below the lowest square, the convention every step of this ratchet has +used. The target the maintainer set on 2026-08-23 was 80. + +Five batches got it there from 75: the WebRTC family, the window backends, the +accessibility backends, the hotkey backends, and the class-callee half of the +adapter sweep. Linux is still the low corner and Windows the high one, for the +reason that has held all along — the facade imports the backend of whatever +platform it is running on, so some of the remaining 18 points cannot be reached +from any single square, and raising this number again means covering code that +runs on all nine. + +### Two Races That Only A Loaded Runner Could Lose + +Both showed up as squares failing in a matrix where the same tests had passed +the round before, which is the shape of a race rather than a regression. + +**A poller that outlived its test.** `AC_usb_watch_start` really starts a +hotplug poller now that the sweep runs genuine objects, and on Windows that +poller shells out to PowerShell every interval. `test_wayland_libei` patches +`subprocess.run` across the whole process and reads the first argv it recorded +— so it read the poller's. Every `_start` in both registries has a `_stop` +sibling, all eight of them, so the sweep now runs it, and an autouse guard +fails any case that leaves a thread behind. The next adapter to grow one is +named here rather than becoming somebody else's flake three files away. + +**A bridge patched on the wrong module.** `send_file` hands the work to +`FileTransferSender`, which reads `get_bridge` out of `webrtc_files`; the host +and viewer fixtures patched it on the host and the viewer. That left the real +bridge on that one path, queueing the chunks onto a background event loop while +the assertion read `sent[0]` immediately afterwards. It passed wherever the +loop won the race — every square, until two lost it in the same round. +`test_webrtc_file_transfer` had always patched the right module, which is why +it never flaked. + +### The AX Constants Came From The Wrong Framework, And Nine Green Squares Said Otherwise + +`Quartz.kAXValueCGPointType` does not exist. pyobjc builds `ApplicationServices` +on top of `HIServices`, which is where the AXValue type constants are declared, +and `Quartz` has no such parent — so on macOS the lookup raised `AttributeError`. +That took out `move()`, `_point()`, and through `_point` the frame comparison +that decides which accessibility element a given Quartz window is, which every +other window action depends on. + +Every square was green while this was true, because the pyobjc stub answered +for the names on `Quartz` — the failure mode the stub was written to create and +`test_pyobjc_stub_names` was written to catch. It caught it on the first run: +the two macOS squares, where the real frameworks are installed, failed on +exactly those two names. The stub now declares them where they live, so they +are checked against `ApplicationServices`. + +The ground truth came out of the wheel rather than a guess: +`HIServices/_metadata.py` defines both constants, and +`ApplicationServices/__init__.py` names `(Quartz, HIServices, CoreText)` as its +parents while `Quartz` names no such thing. + +### When The Callee Is A Class, Build The Real Thing + +The sweep could read a contract made of `Optional[X]`, containers, scalars and +dataclasses. 134 adapters had a callee that is a class, and sat out — the +biggest group left. A constructor's parameters carry annotations too, so an +instance can be built from them the same way a dataclass is built from its +fields; measured, 112 of the 134 are reachable that way. + +What decides whether that is a test or just an execution is which of two shapes +the callee has, and they are handled oppositely: + +**A callee that returns a class** gets a stub returning a real instance. The +adapter then runs its real continuation: `get_egress_policy().is_allowed(url)` +and `parse_baggage(header).to_dict()` execute against the declared shape rather +than against something that answers every method. + +**A callee that is a class** is left alone. Stubbing it was tried first and is +wrong twice over: the prepared instance discards the client's arguments, which +are the thing under test — `CoordinateSpace` divided by a zero `model_w` that no +client sent — and a stub standing in for a class has no alternative +constructors, which surfaced as `'function' object has no attribute 'from_dict'`. +Letting the adapter build the genuine object out of the genuine arguments has +neither problem. + +Running real objects rather than stand-ins had three consequences worth naming: + +**A declared object is no longer flattened to `{}`.** A tool that names `kind` +as required is describing a payload no client would send empty; handing the +adapter `{}` tested the sample rather than the wiring. Samples now follow the +schema's own `properties`, bounded by depth. + +**A class defined beside a third-party import stays out.** `S3ArtifactStore` +builds from its annotations perfectly well and reaches for `boto3` on the first +method call. That is the existing "the adapter is choosing its own backend" +rule one level down, and applying it there kept eight entries that would all +have read "boto3" off the documented-exception list. + +**Both sweeps now run in a directory of their own.** A checkpoint store handed +the sample path creates a SQLite database where it stands: unisolated, the +sweep left `sample.txt` and `value-for-db` in the repository, and one case +passed or failed depending on whether another had run first. + +586 MCP adapters (from 554) and 537 executor adapters (from 471) now get +called. Four need more than any annotation can promise and are named with a +reason each — a presence registry that is real and empty and correctly refuses +an unknown viewer, a cassette with nothing recorded in it, and an anchor whose +`kind` enum sends the call into OpenCV template matching against a real screen. + +### A Whole Subsystem Was Being Measured At Zero, And Its Tests Were Skipping + +`quality.yml` now installs the `[webrtc]` extra. The coverage number was the +smaller half of the reason. Eleven modules under `utils/remote_desktop` raise +`ImportError` at module level without `aiortc`/`av` — 2,090 statements that were +a hard 0% on every square no matter what anyone wrote — but the part that +mattered is that the tests covering the WebRTC host's auth, TLS, resume tokens +and file transfer *were already written*. They `importorskip`ped straight past on +all nine squares, so they ran on developer machines and nowhere else. + +Measured with one variable changed, same machine and same suite: 513 of those +statements are covered by tests that exist today, worth +1.23 points end to end. +The extra resolves on every square (`aiortc` 1.15.0 and `av` 17.1.0 have wheels +for win_amd64, manylinux x86_64 and macos-14 arm64 on both ends of the supported +Python range). `typing-stable-api` deliberately does not get it: that gate's +verdict must not depend on what happens to be installed. + +`dev_requirements.txt` and the `CLAUDE.md` setup line carry it too, because a +developer without it measures about 4 points below the floor CI enforces. + +### The Stub Grows One Level Deeper, And A Third Registry Gets Swept + +Yesterday's sweep replaced each adapter's callee with a value built from its +return annotation. Three things it could not reach, and now does: + +**A dataclass return is not the end of the contract, it is one more level of +it.** `Optional[X]` → None and containers → empty containers stopped at the +first `-> HealOutcome`, so 92 adapters sat out. The stub is now an instance +built from the dataclass's own field annotations, which means the adapter's +`.to_dict()` runs too — against the declared shape rather than a mock that +answers everything. + +**Some adapters import a singleton, not a function.** `default_observer`, +`default_scheduler`, `registry` — the adapter calls one method on the object. +That is the same wiring shape one indirection along, so the callee is the method +and its annotation is the contract; 101 more adapters. Calling *two* methods +means the adapter orchestrates the object rather than standing in front of one +call, and those stay out. + +**The REST route table is the third registry of this shape.** Its handlers' +own module docstring says they are "pure … trivial to unit-test without an HTTP +layer", and the existing REST tests go through the HTTP layer instead — so on a +headless runner most of them reached a handler only to watch it fall into its +own `except` and answer 500. `test_rest_route_sweep.py` calls all 31 routes with +the arguments their own OpenAPI document declares, and asserts what the +dispatcher relies on: `(status, dict)`, a real HTTP code, a payload `json.dumps` +accepts — for a documented request, for the empty one an unhelpful client sends, +and for a body of the wrong shape. It also compares the route table against the +document, so a route nobody describes or a documented route nobody serves is +now a named failure. + +The machinery the three sweeps share moved to `test/unit_test/headless/ +_contract_sweep.py`. Each keeps its own argument source — a JSON schema, the +Script Builder's field specs, an OpenAPI document — which is what stops a sweep +from passing by restating the code it checks. + +Two things fell out of building it. `ac_rrule_next` and its neighbours now +declare `"format": "date-time"` on the properties they parse, which their +descriptions already said in prose and their schemas did not; a client +generating values from the schema alone used to get a `ValueError` out of +`datetime.fromisoformat`. And `AddressBook.set_tags()` cleaned its input with +`str(t).strip()`, so a JSON `null` — what a client sends for an omitted tag — +became a tag literally named `"None"`, which `all_tags()` then listed next to +the real ones. + +### The Trust Store And The Auth Boundary Get Tests, Because Now They Can + +Four modules decide who may drive this machine unattended, whether the host +answering is the one that answered last time, what the viewer reconnects to, and +how hard the encoder is pushed when the link degrades. All are ordinary Python — +a JSON file, a lock and some arithmetic — and none was imported by any test on +any square, because the subsystem could not be loaded without the extra. + +117 tests over the decisions they make in the operator's absence: a trust entry +that must not lose its label on re-add, a store that opens empty rather than +throwing on a truncated file, a fingerprint comparison that survives an SDP +spelling the same certificate in a different case, a token accepted only when it +is an equal string, an IP whitelist that matches by network rather than by +string, a grace period that closes a peer which never authenticated, and five +derived rates that must refuse to invent a number from one sample, a zero +interval, or a counter that went backwards. + +The auth host double supplies exactly the attribute list `ViewerAuthMixin`'s own +docstring asks for, so a mixin that starts reaching for something else fails +there rather than leaning on whatever the real host happens to own. + +### The Two Big WebRTC Classes Did Not Need Splitting, They Needed Doubles + +`Progress.md` had the next step down as "first see whether a testable half can +be split out of `webrtc_host` and `webrtc_viewer`", on the precedent of the two +mixins that had already been carved off them. Going and looking says that was +the wrong question. Neither constructor touches aiortc — the host stores a +config, a rate limiter and a permission set, the viewer stores callbacks — and +every collaborator either arrives as a keyword argument or is a module-level +name. What held them at 19.83% and 14.08% was never the shape of the class; it +was that reaching line one meant standing up an `RTCPeerConnection`, a screen +grabber and a background event loop. + +So the five remaining WebRTC modules got 363 tests against doubles, and the +classes were left where they are: + +| module | before | after | +| --- | ---: | ---: | +| `webrtc_host.py` | 19.83% | **100%** | +| `webrtc_viewer.py` | 14.08% | **100%** | +| `multi_viewer.py` | 21.24% | **100%** | +| `webrtc_audio.py` | 0.00% | **96.27%** | +| `webrtc_transport.py` | 27.89% | **88.84%** | + +Two remainders are deliberate. `_get_cursor_position` is three platform +branches of which any one square runs one, and the two swallowed queue races in +`_enqueue` need a queue that lies about being full — that would be a test of the +double, not of the code. + +Most of what the tests pin down is refusal, because that is most of what these +classes do with what arrives from the wire. The host's four channels each open +with the PeerConnection, which is *before* the token has been checked, so each +one is tested for what it does with traffic from a peer that never +authenticated, from one the operator has since put in read-only, and from one +that is simply flooding. The three inbox verbs — list, fetch, delete — are +tested for the thing they all route through: `_safe_basename`, which is what +stands between a viewer sending `../secret.txt` and the host's own files. + +On the viewer side the load-bearing detail is that slots are found by m-line +order rather than by direction: aiortc gives every answerer transceiver the +default `recvonly` regardless of what the offer asked for, so "my screen goes in +the second video transceiver" is arithmetic, and off by one there replaces the +picture the viewer is watching. Same for the toggles, which are asymmetric on +purpose — off is `replaceTrack(None)` in place, on always renegotiates, because +the host needs a fresh `track` event to restart its consume task. + +The doubles live in `test/unit_test/headless/_webrtc_doubles.py`, shaped like +`_contract_sweep.py` and shared by six test modules. `FakePeerConnection` +carries both ends' surface in one class deliberately: it stands in for one +aiortc type, and splitting it by direction would be two doubles drifting apart +from the same original. + +The thing that had to be split, in the end, was the test files — `CLAUDE.md`'s +750-line limit has no exception for new ones. Host: session (557) and channels +(739). Viewer: session (466), media (385), control (519). + +### The Platform Backends Were Never Linux-Only To Test, They Just Had No Double + +`Progress.md` listed `x11_backend.py` and its accessibility neighbour as +reachable by "only the two Linux squares", and priced them accordingly: work +that lifts two squares out of nine, when the floor is the lowest one. + +That was wrong, and one command says so: + +```bash +python -c "import je_auto_control.wrapper.window_backends.x11_backend" +``` + +That passes on Windows, with no python-Xlib installed. Every `import Xlib`, +`import Quartz`, `import AppKit`, `import ApplicationServices` and +`import comtypes` under `wrapper/window_backends/` is *inside* a method — the +seam was built that way on purpose, so a platform without a backend still +imports. What kept those modules at 0% was never the platform. It was that +nobody had written the double. + +With the frameworks stubbed into `sys.modules`, the whole package goes from +14.24% to 100% and does it on all nine squares: + +| module | before | after | +| --- | ---: | ---: | +| `x11_backend.py` | 0.00% | **100%** | +| `macos_backend.py` | 0.00% | **100%** | +| `__init__.py` (backend selection) | 46.34% | **100%** | +| `base.py` | 63.64% | **100%** | +| `windows_backend.py` | 89.19% | **100%** | + +184 tests, and what they are about is protocol arithmetic that fails quietly. +EWMH requests are addressed to the *root* window with `SubstructureRedirect`, +because that is what routes them to the window manager — sent to the window +itself they reach nobody. `_NET_CLIENT_LIST_STACKING` is bottom-to-top, so it +is reversed and `_NET_CLIENT_LIST`, which carries no order at all, is not. +`move()` sizes the client while `window_rect()` reads the frame, so the +decorations come off the requested size — get that wrong and a window shrinks +by a title bar every time a script round-trips it. On macOS an AX call +returns an error code where zero means success, and `close()` returns +`not error`: read backwards, that is a cheerful "yes" for every action that +failed. + +**A double that nobody checks is a test agreeing with itself**, so the two +stubs are pinned differently, because their constants are different in kind: + +* The Xlib stub's constants carry their real `X.h` values — + `SubstructureRedirectMask` is `1 << 20` — because those numbers go on the + wire. `test_xlib_stub_values.py` compares all twelve against the installed + library wherever there is one, which on CI is both Linux squares. (They + were also checked here against python-Xlib 0.33 pulled into a throwaway + `--target` directory: twelve for twelve.) +* The pyobjc stub's constants are sentinels, because every one of them is + either a key into an info dictionary the stub itself builds or a token + handed straight back to a function the stub itself provides — no number + reaches any arithmetic. `test_pyobjc_stub_names.py` pins the *names* + against the real frameworks on the macOS squares instead, plus the three + window-list flags that genuinely get OR-ed together. + +One thing worth knowing before writing the next one of these: `from +je_auto_control.windows.window import windows_window_manage` resolves by +**attribute on the package** before it consults `sys.modules`. Registering the +double under its own dotted name does nothing on a machine where the real +module is importable — the package it is read off has to be the double. + +### Every Backend Seam Is Covered Now, And The Last One Was Hiding A Leak + +`utils/hotkey/backends/` was the last of the project's backend seams with a +real hole in it — the accessibility, window, OCR, vision, LLM and agent seams +are all covered. Three platforms, three completely different mechanisms: +`RegisterHotKey` and a message pump, `XGrabKey` on the root window, a +`CGEventTap` on a run loop. All three go to 100%, from 12.64%, 24.46% and +40.94%. + +None of them needed a desktop. The Windows backend takes `user32` as an +*argument* to the three methods that do the work, so a recorder drives them +from anywhere; only the prologue that builds it needs `ctypes.wintypes`, and +those two tests say so. The other two import their platform libraries inside +their loops. + +**Changing a hotkey's combo on X11 left the old key grabbed for the life of +the daemon.** `_sync_one` dropped the previous registration from its own +table and never called `ungrab_key`, so the old combo stayed grabbed on the X +server: swallowed from every application, firing nothing, and impossible for +`_ungrab_all` to release at shutdown because it no longer knew about it. +Rebind `ctrl+alt+k` and `ctrl+alt+k` is dead system-wide until the process +exits. + +The same module already knew the shape of that mistake — the rollback in +`_grab_masked` carries a comment saying that leaving grabs held with +`_registered` never updated "leaks the grab and spams BadAccess on every +following poll" — and the Windows backend unregisters at exactly this point. +Only the rebinding path was missed. + +The X11 tests are also what grew the Xlib stub from 12 constants to 18 plus a +keysym table, all of them still compared against the installed library on the +Linux squares (and verified here against python-Xlib 0.33: 31 for 31). + +### The Accessibility Backends, Including The One Nobody Could Reach + +`utils/accessibility` was the second-largest gap in the project and the one +that looked hardest: a UIAutomation provider, an AT-SPI bus and a granted +macOS Accessibility permission are three things no CI runner has, and the +Windows backend alone is 569 statements at 17%. + +The same fact that unlocked the window backends applies here — every +platform import is inside a method — so the three backends and everything +under them now run on all nine squares: + +| module | before | after | +| --- | ---: | ---: | +| `backends/windows_backend.py` | 17.72% | **99.42%** | +| `backends/linux_backend.py` | 42.41% | **100%** | +| `backends/macos_backend.py` | 0.00% | **100%** | +| `backends/windows_query.py` | 24.75% | **99.01%** | +| `backends/windows_state.py` | 18.75% | **100%** | +| `backends/base.py` | 74.03% | **100%** | +| `backends/__init__.py` | 48.94% | **100%** | + +Two of the doubles are worth describing because of what they refuse. + +The UIA one models the indirection every control pattern goes through — ask +an element for a pattern id, then query an interface name off the generated +module — and **refuses a mismatched pair**. Those are two independent +constants with nothing checking them at runtime, and asking for the +ValuePattern id while querying the RangeValuePattern interface fails as a +`None` that reads exactly like "no such control". It also raises +`AttributeError` for any `Current…` property the test did not supply, because +answering with a callable would let `str(pattern.CurrentValue or "")` pass +against the repr of a function. + +The AT-SPI one replaces `SessionBus` rather than `_AtspiConnection`. The +existing Linux test replaces the connection, which is right for testing the +*walk* and leaves the entire D-Bus call layer beneath it unexecuted — and +that is where the protocol lives: the accessibility bus is not the session +bus, an accessible is addressed by a `(sender, path)` *pair*, and the state +bitfield arrives as two 32-bit words, so reading only the first drops every +state above bit 31. + +### 37 Guards That Did Not Contain The Failure They Were Written For + +Writing those tests turned one up. `comtypes` reports a provider failure as +`COMError`, which derives straight from `Exception`: + +```python +>>> issubclass(COMError, (OSError, AttributeError, ValueError, TypeError)) +False +``` + +`windows_query._uia_errors()` exists to say exactly that, and its docstring +names the case: "a window that closes mid-walk, or an application that stops +responding, surfaces exactly that way". Two guards in +`backends/windows_backend.py` used it. The other **37** — every control +pattern in the file — spelled `(OSError, AttributeError, …)` and contained +none of them. + +The race is real and small: `_find_raw` contains a dead provider and answers +`None`, so the found path is the exposed one. Between the search that returns +an element and the `GetCurrentPattern` call that reads it, an application can +go away — and the answer was a `COMError` out of the `ac_*` tool, past the +executor's `AutoControlException` boundary, instead of the `None` the method +promised. All 37 now share one tuple. It only widens what is caught. + +### The Viewer Logged Every Clean Disconnect As An Unhandled Task Exception + +`WebRTCDesktopViewer._consume_video` caught `(OSError, RuntimeError)`. aiortc +ends a track by raising `MediaStreamError` out of `recv()`, and that derives +straight from `Exception`, so it matched neither arm. Nothing awaits that task, +which means the ordinary end of a session — the host stopping its screen share, +or the connection closing — surfaced as asyncio's "Task exception was never +retrieved" instead of the "video stream ended" line the host's own drain loop +and the Opus receiver have always logged. It now catches it, and +`CancelledError` still propagates as the comment there requires. + +The doubles found it: the shared `FrameTrack` ends a stream the way aiortc does, +and the viewer's own test had been written against a local double that ended it +with `OSError`. + +### The Floor Is 75, And The Comment Now Says Which Number That Is + +The nine-way matrix runs 75.79% (ubuntu-22.04 / 3.14) to 76.99% (windows-2022), +up from 69.67–70.97, so `fail_under` goes 69 → 75 on the usual convention: floor +of the lowest square. + +`Progress.md` had recorded that the floor "had to be dug out of the XML +artifact". That is wrong, and following it would set a floor the suite cannot +clear. `coverage report` — the step that enforces `fail_under` — includes branch +coverage, because `branch = true`; Cobertura's `line-rate` attribute does not. +On the same square and the same run those differ by about 2 points (75.79% +against 77.78%). The artifact is the right thing to read for a single +subsystem's gap and the wrong thing to set a floor from. + +## What's new (2026-08-23) + +### A Thousand Adapters Now Get Called, Because the Type Contract Can Build Their Stubs + +`utils/mcp_server/tools/_handlers.py` and the `AC_*` dispatch table in +`utils/executor/action_executor.py` are the same layer twice: about a thousand +short functions whose whole job is to take a client's arguments, call one +headless function, and hand back something that survives `json.dumps`. They were +also the two least-covered files in the project — `_handlers.py` at 37.47%, +`action_executor.py` at 55.78% — and not through neglect. Each adapter is two to +eight lines, so the per-feature test that touches one is testing the feature; the +adapter itself, which is where the wiring lives, was checked by nobody. Wiring +fails only when a client calls it. + +Both registries are now swept, in one file: +`test/unit_test/headless/test_adapter_registry_sweep.py`. 657 MCP tools driven +with the arguments their own JSON schema declares, and 773 `AC_*` commands driven +with the arguments the Script Builder's `command_schema.py` says a client sends. +Neither sweep reads an adapter's source to decide what to pass it, so neither can +pass by restating the code it is checking. + +**What made this possible was the other gate that landed last week.** The +problem with calling a thousand adapters is what to do about the thing each one +calls: a real call moves a mouse, and hand-writing a fake per callee is a +thousand guesses at what each returns. Since the typing contract's exemption +list was emptied on 2026-08-22, there is a third option — every callee's return +annotation is machine-readable, so the stub can be *derived* from it. +`Optional[X]` becomes None, containers become empty containers, scalars become +their zero. Each adapter then runs against exactly what its callee promises, no +more and no less, with no mouse, no display and no network in the picture. + +That is the sharper test as well as the cheaper one: an adapter that needs *more* +than the contract offers now says so here instead of in front of a client. Three +do, and each is named in the file with its reason — one uses the return value as +a context manager, one imports a class rather than a function, one indexes a key +out of a `Dict[str, Any]`. A fourth test fails if any of the three starts +passing, so the list cannot quietly rot. + +| | before | after | +| --- | ---: | ---: | +| `_handlers.py` | 37.47% | **75.21%** | +| `action_executor.py` | 55.78% | **73.52%** | +| whole package | 71.91% | **74.91%** | + +Everything was green on the first run, so these are guards rather than a bug +report — and a guard is worth only what it catches, so it was checked by breaking +it: swapping two arguments inside one adapter (`_open_path(verb, target)`) fails +the forwarding sweep by name, pointing at the tool and both values. + +**One invariant did fall out of building it.** 31 mandatory executor parameters +have no field in the visual editor, and every one of them is annotated `Any`, +`List[...]` or `Dict[...]` — a shape the editor's scalar field types cannot +express, filled from its raw JSON view instead. That was true by convention and +enforced by nobody. A mandatory *scalar* with no field is a different thing +entirely: it means the editor emits an action that raises `TypeError` the first +time it is run. That is a test now, alongside two more of the same kind — every +schema field names a parameter its command accepts, and every command the editor +can emit is a command the dispatch table resolves. + +Out of scope by design, and stated in the file rather than left to be +rediscovered: an adapter that reaches into two project modules composes them +rather than normalising one, and an adapter that imports a third-party module is +picking its own backend, so what it returns depends on the machine — the +opposite of what a sweep can assert. + ## What's new (2026-08-21) ### Two Quality Gates That Had Been Standing Still diff --git a/architecture_explore.md b/architecture_explore.md index b3438b8a..25a3f624 100644 --- a/architecture_explore.md +++ b/architecture_explore.md @@ -6,7 +6,7 @@ > 擷取每個模組的 docstring 與頂層公開名稱；統計數字取自實際檔案，非估算。 > 指令數與公開 API 數以 `executor.known_commands()` 與 `je_auto_control.__all__` 在工作樹上實測取得。 > -> **掃描時間**：2026-08-21　**版本**：`pyproject.toml` version `0.0.220`　**分支**：`feat/typing-contract-and-coverage-ratchet` +> **掃描時間**：2026-08-24　**版本**：`pyproject.toml` version `0.0.221`　**分支**：`feat/coverage-to-80` --- @@ -20,7 +20,7 @@ iOS（WebDriverAgent）。核心能力是滑鼠／鍵盤控制、影像辨識、 | 指標 | 數值 | | --- | ---: | | Python 模組總數（含周邊子專案） | 1,032 | -| 程式碼總行數 | 141,316 | +| 程式碼總行數 | 141,350 | | `je_auto_control/utils/` 子套件數 | 310 | | `AC_*` 動作指令數（`known_commands()` 實測） | 773 | | 套件門面 `__all__` 公開名稱數 | 1,238 | @@ -179,7 +179,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `wrapper/auto_control_image.py` | 83 | 影像 API：`locate_all_image`、`locate_image_center`、`locate_and_click`。 | | `wrapper/auto_control_record.py` | 114 | 錄製 API：`record`／`stop_record`／`record_to_json`（支援 stop event 與逾時）。 | | `wrapper/auto_control_window.py` | 278 | 視窗管理門面：列舉、尋找、聚焦、等待、關閉、顯示狀態、幾何、所屬行程 PID、依行程列舉／最小化視窗、不搶焦點的投遞式輸入（目前僅 Windows 實作）。 | -| `wrapper/window_backends/` | 985 | 視窗管理的平台縫（`base` / `windows_backend` / `x11_backend` / `macos_backend` / `null_backend`）。放在 `wrapper/` 而不是 `utils/`，因為它必須 import `windows/`、`linux_with_x11/`、`osx/`，而 `utils/` 在分層上在那三者之上。 | +| `wrapper/window_backends/` | 988 | 視窗管理的平台縫（`base` / `windows_backend` / `x11_backend` / `macos_backend` / `null_backend`）。放在 `wrapper/` 而不是 `utils/`，因為它必須 import `windows/`、`linux_with_x11/`、`osx/`，而 `utils/` 在分層上在那三者之上。 | ### 5.3 平台後端 @@ -320,11 +320,11 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.3 排程、觸發與背景監看 -> 11 個套件、約 3,554 行。 +> 11 個套件、約 3,562 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | -| `utils/hotkey/` | 727 | 全域熱鍵守護行程，把 OS 層熱鍵綁到 action 檔（Win/macOS/X11 三後端） | +| `utils/hotkey/` | 735 | 全域熱鍵守護行程，把 OS 層熱鍵綁到 action 檔（Win/macOS/X11 三後端） | | `utils/idle_keepawake/` | 212 | 偵測使用者閒置時間並在無人值守執行期間阻止系統睡眠 | | `utils/lock_session/` | 163 | 鎖定工作站、等待解鎖並分類鎖定狀態轉換 | | `utils/observer/` | 220 | 反應式畫面觀察者，在出現／消失／變化時觸發 | @@ -437,12 +437,12 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.7 無障礙樹與原生控制項 -> 16 個套件、約 4,313 行。 +> 16 個套件、約 4,318 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | | `utils/a11y_audit/` | 355 | 以無障礙樹 + OCR 進行無障礙與 i18n 稽核 | -| `utils/accessibility/` | 2,835 | 跨平台無障礙樹定位與錄製；Windows UIA／macOS AX／null 三後端。支援限定視窗（換搜尋起點，不是過濾）、逐節點可中斷走訪、`IUIAutomation2` 連線逾時、名稱子字串比對與排序、`control_get_state` 一次讀完值／勾選／選取／數值（密碼欄位不回內容） | +| `utils/accessibility/` | 2,840 | 跨平台無障礙樹定位與錄製；Windows UIA／macOS AX／null 三後端。支援限定視窗（換搜尋起點，不是過濾）、逐節點可中斷走訪、`IUIAutomation2` 連線逾時、名稱子字串比對與排序、`control_get_state` 一次讀完值／勾選／選取／數值（密碼欄位不回內容） | | `utils/ax_events/` | 29 | 反應式 UIA 事件等待（focus-changed） | | `utils/ax_props/` | 44 | 讀取豐富 UIA 屬性（enabled／offscreen／help／status／快捷鍵） | | `utils/ax_text/` | 102 | 透過 UIA TextPattern 取得原生文字（讀取／尋找／選取／屬性） | @@ -490,7 +490,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 ### 5.4.9 AI / Agent / LLM -> 13 個套件、約 20,643 行。 +> 13 個套件、約 20,645 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | @@ -503,21 +503,21 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `utils/cua_action/` | 127 | 標準化 computer-use 動作結構（Anthropic／OpenAI → `AC_*`） | | `utils/llm/` | 357 | 自然語言 → action list 規劃器 + Anthropic／null 後端 | | `utils/mcp_registry/` | 92 | MCP registry `server.json` 資訊清單產生（可被發現） | -| `utils/mcp_server/` | 17,354 | **無頭 MCP 伺服器**（16K LOC，預設註冊 676 個工具＝657 個 `ac_*` + 19 個別名）：stdio + HTTP 傳輸、工具工廠與處理器、資源、prompt、稽核、限流、外掛熱重載 | +| `utils/mcp_server/` | 17,356 | **無頭 MCP 伺服器**（16K LOC，預設註冊 676 個工具＝657 個 `ac_*` + 19 個別名）：stdio + HTTP 傳輸、工具工廠與處理器、資源、prompt、稽核、限流、外掛熱重載 | | `utils/tool_use_schema/` | 180 | 把 `AC_*` 指令匯出成 Claude／OpenAI 的 tool-use schema | | `utils/trajectory_eval/` | 106 | agent 軌跡評估：依評分規準為一次執行打分 | | `utils/vision/` | 448 | VLM 元素定位器（依描述找元素）+ Anthropic／OpenAI／null 後端 | ### 5.4.10 遠端桌面與 USB -> 6 個套件、約 17,903 行。 +> 6 個套件、約 17,919 行。 | 模組 | 行數 | 職責 | | --- | ---: | --- | | `utils/admin/` | 328 | 多主機管理主控台：平行輪詢 N 個 AutoControl REST 端點 | | `utils/config_sync/` | 246 | 透過訊令伺服器做跨機器設定同步 | | `utils/device_matrix/` | 138 | 行動裝置矩陣：同一 action list 於多台裝置平行執行 | -| `utils/remote_desktop/` | 11,990 | **遠端桌面子系統**（56 檔／11.7K LOC）：TCP／WebSocket／WebRTC 三條傳輸路徑、主機與檢視端、訊令伺服器、TURN／中繼、多檢視者、錄影、信任清單、TOTP、稽核鏈 | +| `utils/remote_desktop/` | 12,006 | **遠端桌面子系統**（56 檔／11.7K LOC）：TCP／WebSocket／WebRTC 三條傳輸路徑、主機與檢視端、訊令伺服器、TURN／中繼、多檢視者、錄影、信任清單、TOTP、稽核鏈 | | `utils/usb/` | 4,281 | 跨平台 USB 列舉／熱插拔／裝置直通（WinUSB、IOKit、libusb 後端 + ACL + WebRTC DataChannel 通道） | | `utils/usbip/` | 920 | USB/IP 線路協定主機端（協定封包、TCP 伺服器、libusb URB 後端） | @@ -702,7 +702,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `action_schema.py` | 128 | action list 的結構驗證：形狀、參數型別、未知指令拒絕。單一走訪同時支援兩種消費方式：`validate_actions()` 遇到第一個問題就拋、`unknown_command_names()` 收齊全部不認得的名字（REST `/execute` 用它回 400）。 | | `mouse_aliases.py` | 39 | 單鍵點擊別名（`AC_click_left` 等），executor 與 callback executor 共用。 | -#### `utils/mcp_server/`（17,354 行，676 個工具）— 最大子系統 +#### `utils/mcp_server/`（17,356 行，676 個工具）— 最大子系統 | 檔案 | 行數 | 職責 | | --- | ---: | --- | @@ -726,14 +726,14 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `rate_limit.py` | 48 | 工具呼叫的 token bucket 限流。 | | `__main__.py` | 88 | `je_auto_control_mcp` console script 進入點。 | -#### `utils/remote_desktop/`（11,990 行／56 檔） +#### `utils/remote_desktop/`（12,006 行／56 檔） 三條傳輸路徑並存：**TCP**（JPEG 影格）、**WebSocket**（同協定換傳輸）、**WebRTC**（aiortc 視訊 + DataChannel）。 | 檔案 | 行數 | 職責 | | --- | ---: | --- | | `webrtc_host.py` | 702 | WebRTC 主機：串流螢幕視訊並接受檢視端輸入;session 生命週期、DataChannel 接線、檔案收發。 | -| `webrtc_viewer.py` | 662 | WebRTC 檢視端：接收視訊並送出輸入。 | +| `webrtc_viewer.py` | 672 | WebRTC 檢視端：接收視訊並送出輸入。 | | `host.py` | 625 | TCP 主機：接受迴圈、TLS 包裝、連線／認證握手、音訊與剪貼簿廣播、檔案推送、單次 token。 | | `viewer.py` | 623 | TCP 檢視端。 | | `host_service.py` | 542 | 無頭 WebRTC 主機執行器 + 多平台服務安裝器。 | @@ -751,7 +751,7 @@ socket server 有 8 MiB 讀取上限與 30 秒 handler timeout。 | `turn_config.py` | 234 | coturn 設定產生器。 | | `presence.py` | 221 | 多檢視者的執行緒安全在場註冊表。 | | `jpeg_recorder_encrypted.py` | 223 | AES-GCM 加密版 session 錄影。 | -| `address_book.py` | 209 | 檢視端的主機通訊錄。 | +| `address_book.py` | 215 | 檢視端的主機通訊錄。 | | `audio.py` / `webrtc_audio.py` / `webrtc_mic.py` | 206 / 190 / 152 | 音訊擷取播放、音訊軌、麥克風上行。 | | `webrtc_files.py` | 205 | 專屬 DataChannel 的分塊檔案傳輸。 | | `webrtc_host_auth.py` | 222 | 檢視端認證與核准：token 檢查、信任清單／IP 白名單自動放行、手動接受／拒絕、SAS、逾時關閉。 | @@ -1026,13 +1026,13 @@ socket 預設綁 `127.0.0.1`；資源一律用 `with`。 | 層／子系統 | 檔案數 | 行數 | | --- | ---: | ---: | | `gui/` | 90 | 26,699 | -| `utils/mcp_server/` | 21 | 17,354 | -| `utils/remote_desktop/` | 56 | 11,990 | +| `utils/mcp_server/` | 21 | 17,356 | +| `utils/remote_desktop/` | 56 | 12,006 | | `utils/executor/` | 6 | 9,081 | | `utils/usb/` | 17 | 4,281 | | `je_auto_control/`（頂層 3 檔） | 3 | 2,367 | -| `utils/accessibility/` | 13 | 2,835 | -| `wrapper/` | 19 | 3,514 | +| `utils/accessibility/` | 13 | 2,840 | +| `wrapper/` | 19 | 3,517 | | `windows/` | 23 | 1,906 | | `utils/rest_api/` | 8 | 1,751 | | `utils/agent/` | 8 | 1,250 | @@ -1044,7 +1044,7 @@ socket 預設綁 `127.0.0.1`；資源一律用 `with`。 | `utils/assertion/` | 3 | 863 | | `osx/` | 17 | 915 | | `autocontrol-lsp/` | 8 | 744 | -| `utils/hotkey/` | 7 | 727 | +| `utils/hotkey/` | 7 | 735 | | 其餘模組（約 286 個 `utils/` 子套件 + `android/`／`ios/`／周邊小工具） | 673 | 47,683 | -| **總計** | **1,026** | **141,251** | +| **總計** | **1,026** | **141,285** | diff --git a/dev_requirements.txt b/dev_requirements.txt index d64208db..0f26e449 100644 --- a/dev_requirements.txt +++ b/dev_requirements.txt @@ -9,6 +9,12 @@ qt-material==2.17 mss==10.2.0 defusedxml==0.7.1 +# WebRTC ([webrtc] extra) — without these, eleven `utils/remote_desktop` +# modules raise ImportError at import and their tests skip, so the coverage +# figure a developer measures is ~4 points under the one CI enforces. +aiortc>=1.14.0 +av>=14.0.0 + # Office I/O ([office] extra) — exercised by the headless Office tests. openpyxl==3.1.5 python-docx==1.2.0 diff --git a/je_auto_control/utils/accessibility/backends/windows_backend.py b/je_auto_control/utils/accessibility/backends/windows_backend.py index 0d2783f4..a7833263 100644 --- a/je_auto_control/utils/accessibility/backends/windows_backend.py +++ b/je_auto_control/utils/accessibility/backends/windows_backend.py @@ -10,7 +10,7 @@ """ import functools import sys -from typing import Any, Dict, List, Optional +from typing import Any, Dict, List, Optional, Tuple, Type from je_auto_control.utils.accessibility.backends.base import ( AccessibilityBackend, @@ -26,6 +26,11 @@ ) from je_auto_control.utils.logging.logging_instance import autocontrol_logger +#: Every guard below. ``COMError`` is why a narrower tuple would contain +#: nothing — see ``windows_query._uia_errors``; ``TypeError`` joins it because +#: several of these reads coerce whatever the provider returned. +_UIA_ERRORS: Tuple[Type[BaseException], ...] = UIA_ERRORS + (TypeError,) + _UIA_IS_CONTROL_ELEMENT_PROPERTY = 30016 _UIA_NAME_PROPERTY = 30005 _UIA_VALUE_PATTERN_ID = 10002 @@ -98,7 +103,7 @@ def _create_automation(uia_module): interface=interface) automation.ConnectionTimeout = _CONNECTION_TIMEOUT_MS return automation - except (OSError, AttributeError, ValueError) as error: + except _UIA_ERRORS as error: autocontrol_logger.info( "UIAutomation2 unavailable, provider waits are unbounded: %r", error) @@ -258,7 +263,7 @@ def _pattern(self, raw, pattern_id, interface_name): return None interface = getattr(self._uia_module, interface_name) return unknown.QueryInterface(interface) - except (OSError, AttributeError, ValueError): + except _UIA_ERRORS: return None def get_value(self, name=None, role=None, app_name=None, @@ -278,7 +283,7 @@ def get_value(self, name=None, role=None, app_name=None, return None try: return str(pattern.CurrentValue or "") - except (OSError, AttributeError): + except _UIA_ERRORS: return None def set_value(self, value, name=None, role=None, app_name=None, @@ -291,7 +296,7 @@ def set_value(self, value, name=None, role=None, app_name=None, try: pattern.SetValue(str(value)) return True - except (OSError, AttributeError): + except _UIA_ERRORS: return False def invoke(self, name=None, role=None, app_name=None, @@ -304,7 +309,7 @@ def invoke(self, name=None, role=None, app_name=None, try: pattern.Invoke() return True - except (OSError, AttributeError): + except _UIA_ERRORS: return False def toggle(self, name=None, role=None, app_name=None, @@ -317,7 +322,7 @@ def toggle(self, name=None, role=None, app_name=None, try: pattern.Toggle() return True - except (OSError, AttributeError): + except _UIA_ERRORS: return False def read_table(self, name=None, role=None, app_name=None, @@ -330,7 +335,7 @@ def read_table(self, name=None, role=None, app_name=None, try: rows = int(pattern.CurrentRowCount or 0) cols = int(pattern.CurrentColumnCount or 0) - except (OSError, AttributeError): + except _UIA_ERRORS: return [] return [self._read_row(pattern, r, cols) for r in range(rows)] @@ -344,7 +349,7 @@ def _invoke_pattern_method(self, name, role, app_name, automation_id, try: action(pattern) return True - except (OSError, AttributeError): + except _UIA_ERRORS: return False def expand(self, name=None, role=None, app_name=None, automation_id=None): @@ -366,7 +371,7 @@ def expand_state(self, name=None, role=None, app_name=None, return None try: return _EXPAND_STATES.get(int(pattern.CurrentExpandCollapseState)) - except (OSError, AttributeError, ValueError, TypeError): + except _UIA_ERRORS: return None def select_item(self, name=None, role=None, app_name=None, automation_id=None): @@ -397,7 +402,7 @@ def get_range(self, name=None, role=None, app_name=None, return {"value": float(pattern.CurrentValue), "minimum": float(pattern.CurrentMinimum), "maximum": float(pattern.CurrentMaximum)} - except (OSError, AttributeError, ValueError, TypeError): + except _UIA_ERRORS: return None def _realize(self, raw) -> None: @@ -408,7 +413,7 @@ def _realize(self, raw) -> None: return try: pattern.Realize() - except (OSError, AttributeError): + except _UIA_ERRORS: pass def find_virtual_item(self, item_name=None, by="name", container_name=None, @@ -424,7 +429,7 @@ def find_virtual_item(self, item_name=None, by="name", container_name=None, else _UIA_NAME_PROPERTY) try: found = pattern.FindItemByProperty(None, property_id, item_name) - except (OSError, AttributeError, ValueError): + except _UIA_ERRORS: return None if not found: return None @@ -479,7 +484,7 @@ def window_interaction_state(self, name=None, role=None, app_name=None, try: return _WINDOW_INTERACTION_STATES.get( int(pattern.CurrentWindowInteractionState)) - except (OSError, AttributeError, ValueError, TypeError): + except _UIA_ERRORS: return None def legacy_info(self, name=None, role=None, app_name=None, @@ -511,7 +516,7 @@ def get_selection(self, name=None, role=None, app_name=None, items = _header_names(pattern.GetCurrentSelection()) can_multiple = bool(pattern.CurrentCanSelectMultiple) required = bool(pattern.CurrentIsSelectionRequired) - except (OSError, AttributeError): + except _UIA_ERRORS: return None return {"items": items, "can_select_multiple": can_multiple, "is_required": required} @@ -529,7 +534,7 @@ def list_views(self, name=None, role=None, app_name=None, try: view_ids = list(pattern.GetCurrentSupportedViews()) current = int(pattern.CurrentCurrentView) - except (OSError, AttributeError, ValueError, TypeError): + except _UIA_ERRORS: return None return {"current": _view_name(pattern, current), "views": [_view_name(pattern, view_id) for view_id in view_ids]} @@ -544,7 +549,7 @@ def set_view(self, view="", name=None, role=None, app_name=None, if _view_name(pattern, view_id) == str(view): pattern.SetCurrentView(int(view_id)) return True - except (OSError, AttributeError, ValueError, TypeError): + except _UIA_ERRORS: return False return False @@ -577,7 +582,7 @@ def wait_for_focus_change(self, timeout=5.0) -> Optional[Dict[str, Any]]: try: with self._event_lock: automation.AddFocusChangedEventHandler(None, handler) - except (OSError, AttributeError): + except _UIA_ERRORS: return None try: return events.get(timeout=float(timeout)) @@ -587,7 +592,7 @@ def wait_for_focus_change(self, timeout=5.0) -> Optional[Dict[str, Any]]: with self._event_lock: try: automation.RemoveFocusChangedEventHandler(handler) - except (OSError, AttributeError): + except _UIA_ERRORS: pass def get_table_headers(self, name=None, role=None, app_name=None, @@ -600,7 +605,7 @@ def get_table_headers(self, name=None, role=None, app_name=None, try: columns = pattern.GetCurrentColumnHeaders() rows = pattern.GetCurrentRowHeaders() - except (OSError, AttributeError): + except _UIA_ERRORS: return None return {"columns": _header_names(columns), "rows": _header_names(rows)} @@ -613,7 +618,7 @@ def get_grid_cell(self, row=0, column=0, name=None, role=None, return None try: cell = grid.GetItem(int(row), int(column)) - except (OSError, AttributeError): + except _UIA_ERRORS: return None if not cell: return None @@ -636,7 +641,7 @@ def document_text(self, name=None, role=None, app_name=None, return None try: return str(pattern.DocumentRange.GetText(-1) or "") - except (OSError, AttributeError): + except _UIA_ERRORS: return None def selected_text(self, name=None, role=None, app_name=None, @@ -649,7 +654,7 @@ def selected_text(self, name=None, role=None, app_name=None, if not selection or int(selection.Length or 0) == 0: return "" return str(selection.GetElement(0).GetText(-1) or "") - except (OSError, AttributeError): + except _UIA_ERRORS: return None def visible_text(self, name=None, role=None, app_name=None, @@ -662,7 +667,7 @@ def visible_text(self, name=None, role=None, app_name=None, count = int(ranges.Length or 0) return "".join(str(ranges.GetElement(i).GetText(-1) or "") for i in range(count)) - except (OSError, AttributeError): + except _UIA_ERRORS: return None def _find_range(self, text, ignore_case, name, role, app_name, automation_id): @@ -673,7 +678,7 @@ def _find_range(self, text, ignore_case, name, role, app_name, automation_id): try: return pattern.DocumentRange.FindText(str(text), False, bool(ignore_case)) - except (OSError, AttributeError): + except _UIA_ERRORS: return None def find_text(self, text="", ignore_case=True, name=None, role=None, @@ -690,7 +695,7 @@ def select_text(self, text="", ignore_case=True, name=None, role=None, try: found.Select() return True - except (OSError, AttributeError): + except _UIA_ERRORS: return False def text_attributes(self, name=None, role=None, app_name=None, @@ -703,7 +708,7 @@ def text_attributes(self, name=None, role=None, app_name=None, text_range = (selection.GetElement(0) if selection and int(selection.Length or 0) > 0 else pattern.DocumentRange) - except (OSError, AttributeError): + except _UIA_ERRORS: return None return _read_text_attributes(text_range) @@ -715,7 +720,7 @@ def set_focus(self, name=None, role=None, app_name=None, try: raw.SetFocus() return True - except (OSError, AttributeError): + except _UIA_ERRORS: return False @staticmethod @@ -726,7 +731,7 @@ def _read_row(pattern, row: int, cols: int): try: cell = pattern.GetItem(row, col) cells.append(str(cell.CurrentName or "") if cell else "") - except (OSError, AttributeError): + except _UIA_ERRORS: cells.append("") return cells @@ -735,7 +740,7 @@ def _view_name(pattern, view_id) -> str: """Return a MultipleViewPattern view's name, or '' on failure.""" try: return str(pattern.GetViewName(int(view_id)) or "") - except (OSError, AttributeError, ValueError, TypeError): + except _UIA_ERRORS: return "" @@ -744,12 +749,12 @@ def _header_names(array) -> List[str]: names: List[str] = [] try: count = int(array.Length or 0) - except (OSError, AttributeError): + except _UIA_ERRORS: return names for index in range(count): try: names.append(str(array.GetElement(index).CurrentName or "")) - except (OSError, AttributeError): + except _UIA_ERRORS: names.append("") return names @@ -766,7 +771,7 @@ def _read_cell(item_pattern, cell, row: int, column: int) -> Dict[str, Any]: ("column_span", "CurrentColumnSpan")): try: info[key] = int(getattr(item_pattern, attr)) - except (OSError, AttributeError, ValueError, TypeError): + except _UIA_ERRORS: pass return info @@ -774,7 +779,7 @@ def _read_cell(item_pattern, cell, row: int, column: int) -> Dict[str, Any]: def _safe_name(raw) -> str: try: return str(raw.CurrentName or "") - except (OSError, AttributeError): + except _UIA_ERRORS: return "" @@ -793,7 +798,7 @@ def _as_text(value) -> str: def _attr(text_range, attribute_id, cast): try: return cast(text_range.GetAttributeValue(attribute_id)) - except (OSError, AttributeError, ValueError, TypeError): + except _UIA_ERRORS: return None @@ -826,7 +831,7 @@ def _read_legacy(pattern) -> Dict[str, Any]: for key, attribute, cast in _LEGACY_READS: try: info[key] = cast(getattr(pattern, attribute)) - except (OSError, AttributeError, ValueError, TypeError): + except _UIA_ERRORS: info[key] = None return info @@ -849,7 +854,7 @@ def _read_properties(raw) -> Dict[str, Any]: for key, attribute, cast in _PROPERTY_READS: try: properties[key] = cast(getattr(raw, attribute)) - except (OSError, AttributeError, ValueError, TypeError): + except _UIA_ERRORS: properties[key] = None return properties @@ -871,7 +876,7 @@ def _convert_uia(raw, cached: bool = False) -> Optional[AccessibilityElement]: process_id = int(getattr(raw, prefix + "ProcessId") or 0) automation_id = str(getattr(raw, prefix + "AutomationId") or "") enabled = bool(getattr(raw, prefix + "IsEnabled")) - except (OSError, AttributeError): + except _UIA_ERRORS: return None width = max(0, int(rect.right - rect.left)) height = max(0, int(rect.bottom - rect.top)) diff --git a/je_auto_control/utils/hotkey/backends/linux_backend.py b/je_auto_control/utils/hotkey/backends/linux_backend.py index 2066d17a..db061ba2 100644 --- a/je_auto_control/utils/hotkey/backends/linux_backend.py +++ b/je_auto_control/utils/hotkey/backends/linux_backend.py @@ -106,6 +106,14 @@ def _sync_one(self, root, binding: HotkeyBinding) -> None: if prior is not None and prior[0] == binding.combo: return if prior is not None: + # Release the old key before forgetting it. Dropping it from + # `_registered` alone leaves the grab held on the server for the + # life of the process — the *previous* combo keeps being consumed + # from every application and fires nothing, and `_ungrab_all` + # cannot release what it no longer knows about. The Windows + # backend has always unregistered here. + _combo, prior_mask, prior_keycode = prior + self._ungrab_masked(root, prior_keycode, prior_mask) self._registered.pop(binding.binding_id, None) try: mask, keycode = _combo_to_x11(binding.combo) diff --git a/je_auto_control/utils/mcp_server/tools/_factories.py b/je_auto_control/utils/mcp_server/tools/_factories.py index ee2bfc39..82de33ca 100644 --- a/je_auto_control/utils/mcp_server/tools/_factories.py +++ b/je_auto_control/utils/mcp_server/tools/_factories.py @@ -6035,8 +6035,8 @@ def locale_tools() -> List[MCPTool]: description=("Format an ISO (YYYY-MM-DD) date for a locale. 'fmt' " "is short/medium/long/full. Returns {text}."), input_schema=schema( - {"value": {"type": "string"}, "locale": _LOC, - "fmt": {"type": "string"}}, ["value"]), + {"value": {"type": "string", "format": "date"}, + "locale": _LOC, "fmt": {"type": "string"}}, ["value"]), handler=h.format_date, annotations=READ_ONLY, ), @@ -7533,7 +7533,8 @@ def recurrence_tools() -> List[MCPTool]: "'dtstart' into the next 'count' ISO datetimes. " "Returns {occurrences}."), input_schema=schema( - {"rule": {"type": "string"}, "dtstart": {"type": "string"}, + {"rule": {"type": "string"}, + "dtstart": {"type": "string", "format": "date-time"}, "count": {"type": "integer"}}, ["rule", "dtstart"]), handler=h.rrule_occurrences, @@ -7545,8 +7546,9 @@ def recurrence_tools() -> List[MCPTool]: "(ISO; defaults to current time), anchored at ISO " "'dtstart'. Returns {next}."), input_schema=schema( - {"rule": {"type": "string"}, "dtstart": {"type": "string"}, - "now": {"type": "string"}}, + {"rule": {"type": "string"}, + "dtstart": {"type": "string", "format": "date-time"}, + "now": {"type": "string", "format": "date-time"}}, ["rule", "dtstart"]), handler=h.rrule_next, annotations=READ_ONLY, diff --git a/je_auto_control/utils/remote_desktop/address_book.py b/je_auto_control/utils/remote_desktop/address_book.py index 279dbfd1..8727adfd 100644 --- a/je_auto_control/utils/remote_desktop/address_book.py +++ b/je_auto_control/utils/remote_desktop/address_book.py @@ -142,8 +142,14 @@ def _build_entry(*, host_id: str, server_url: str, def set_tags(self, *, host_id: str, server_url: str, tags: list) -> None: - """Replace ``tags`` on the matching entry.""" - clean = [str(t).strip() for t in tags if str(t).strip()] + """Replace ``tags`` on the matching entry. + + A JSON ``null`` reaches here as None, and ``str(None)`` is the + non-empty string ``"None"`` -- so an omitted tag used to be stored as + one, and `all_tags` then listed it alongside the real ones. + """ + clean = [str(t).strip() for t in tags + if t is not None and str(t).strip()] with self._lock: for entry in self._entries: if (entry.get("host_id") == host_id diff --git a/je_auto_control/utils/remote_desktop/webrtc_viewer.py b/je_auto_control/utils/remote_desktop/webrtc_viewer.py index 72ee7331..dbdd1795 100644 --- a/je_auto_control/utils/remote_desktop/webrtc_viewer.py +++ b/je_auto_control/utils/remote_desktop/webrtc_viewer.py @@ -540,6 +540,14 @@ async def _consume_video(self, track) -> None: # CancelledError is intentionally not caught — it must propagate # so the awaiter knows the consumer ended via cancellation # rather than a stream error (S7497). + # + # MediaStreamError is the *normal* end: aiortc raises it from recv() + # when the host stops sharing or the connection closes. It derives + # straight from Exception, so it is not covered by the OSError / + # RuntimeError arm below, and nobody awaits this task — letting it + # escape turns every clean disconnect into an un-retrieved task + # exception. The host's own drain loop has always caught it. + from aiortc.mediastreams import MediaStreamError try: while not self._closed.is_set(): frame = await track.recv() @@ -548,6 +556,8 @@ async def _consume_video(self, track) -> None: self._on_frame(frame) except (RuntimeError, OSError) as error: autocontrol_logger.debug("frame cb: %r", error) + except MediaStreamError: + autocontrol_logger.info("webrtc viewer: video stream ended") except (OSError, RuntimeError) as error: autocontrol_logger.info("webrtc viewer: video stream ended: %r", error) diff --git a/je_auto_control/wrapper/window_backends/macos_backend.py b/je_auto_control/wrapper/window_backends/macos_backend.py index edaf30e0..1aaa5d99 100644 --- a/je_auto_control/wrapper/window_backends/macos_backend.py +++ b/je_auto_control/wrapper/window_backends/macos_backend.py @@ -237,10 +237,14 @@ def move(self, window_id: int, x: int, y: int, window = self._ax_window(window_id) if window is None: return False + # The AXValue type constants come from ApplicationServices, not + # Quartz: pyobjc builds ApplicationServices on top of HIServices, + # which declares them, and Quartz has no such parent. Spelling them + # `Quartz.` raised AttributeError on the one platform this runs on. position = ax.AXValueCreate( - Quartz.kAXValueCGPointType, Quartz.CGPoint(float(x), float(y))) + ax.kAXValueCGPointType, Quartz.CGPoint(float(x), float(y))) size = ax.AXValueCreate( - Quartz.kAXValueCGSizeType, + ax.kAXValueCGSizeType, Quartz.CGSize(float(width), float(height))) moved = ax.AXUIElementSetAttributeValue(window, "AXPosition", position) resized = ax.AXUIElementSetAttributeValue(window, "AXSize", size) @@ -279,11 +283,10 @@ def _best_match(candidates: list, wanted_origin: Tuple[int, int], def _point(value: Any) -> Tuple[int, int]: """Read an ``AXValue`` point as ``(x, y)``, or ``(-1, -1)``.""" import ApplicationServices as ax - import Quartz if value is None: return (-1, -1) - ok, point = ax.AXValueGetValue(value, Quartz.kAXValueCGPointType, None) + ok, point = ax.AXValueGetValue(value, ax.kAXValueCGPointType, None) if not ok or point is None: return (-1, -1) return (int(point.x), int(point.y)) diff --git a/pyproject.toml b/pyproject.toml index e4150f14..1e81f484 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -172,9 +172,36 @@ precision = 2 # Floored to the integer below the lowest square, the same convention that # turned a 50.26% minimum into 50. # -# 70 was the destination this was climbing towards and the suite is level with -# it; Progress.md records that the next destination is the maintainer's call. -fail_under = 69 +# 70 was the destination that number was climbing towards; the maintainer set +# the next one at 80 on 2026-08-23, and Progress.md tracks the climb. +# +# Measured 2026-08-24 on the last run for this PR, after `quality.yml` began +# installing the `[webrtc]` extra and the WebRTC subsystem got its first +# tests: the nine-way matrix runs 75.79% (ubuntu-22.04 / 3.14) to 76.99% +# (windows-2022, 3.11 and 3.12). Windows is still the high corner for the same +# reason as before — the facade imports the backend of the platform it is +# running on. Floored to the integer below the lowest square: 75. +# +# Read that matrix from the "Enforce the coverage floor" step, which is this +# report and therefore the number `fail_under` is compared against. The +# `coverage.xml` artifact is NOT interchangeable with it: `branch = true` +# above puts branches in this total and Cobertura's `line-rate` attribute +# leaves them out, so the XML reads about 2 points higher (77.78% against +# 75.79% on ubuntu-22.04 / 3.14, same run). A floor set from the artifact +# would be a floor the suite cannot clear. +# +# Measured 2026-08-24 on the run that turned every square green, after the +# WebRTC modules, the window and accessibility backends, the hotkey backends +# and the class-callee half of the adapter sweep: the nine-way matrix runs +# 81.40% (ubuntu-22.04 / 3.14) to 82.73% (windows-2022 / 3.12). That clears +# the 80 the maintainer set on 2026-08-23. Floored to the integer below the +# lowest square, the same convention every step of this ratchet has used: 81. +# +# The spread is still about 1.3 points and Windows is still the high corner, +# for the reason given above — the facade imports the backend of the platform +# it is running on, so part of the remainder is unreachable from any single +# square. Raising this number again means covering code that runs on all nine. +fail_under = 81 [tool.mypy] python_version = "3.10" diff --git a/test/unit_test/headless/_contract_sweep.py b/test/unit_test/headless/_contract_sweep.py new file mode 100644 index 00000000..9b403660 --- /dev/null +++ b/test/unit_test/headless/_contract_sweep.py @@ -0,0 +1,535 @@ +"""The machinery three sweeps share: what a callee promises, and how to fake it. + +``test_adapter_registry_sweep`` (MCP tools and the ``AC_*`` table) and +``test_rest_route_sweep`` (the REST route table) all face the same problem. +Each registry is a few hundred short functions whose whole job is to take a +client's arguments, call one headless function, and hand back something that +survives ``json.dumps``. Calling one for real needs a mouse, a screen or a +network; not calling it at all leaves the wiring checked by nobody. + +The way out is the same in every case: **replace the callee with a stub built +from its own return annotation**. That became possible when the typing +contract's exemption list was emptied on 2026-08-22 -- before that, 136 modules +had nothing to read. "What is this adapter entitled to assume?" is now a +question a program can answer, so an adapter can be run against exactly what +its callee promises, no more and no less. + +This module holds only that shared half: reading a value out of a declared +type, finding the callee an adapter stands in front of, and installing the +stub. Each sweep keeps its own argument source -- a JSON schema, the Script +Builder's field specs, an HTTP request context -- because that is what stops a +sweep from passing by restating the code it checks. + +Nothing here is a test; the file is named so pytest does not collect it. +""" +import ast +import collections.abc +import dataclasses +import functools +import importlib +import inspect +import json +import sys +import textwrap +import typing +from typing import Any, Callable, Dict, FrozenSet, List, Optional, Tuple + +import pytest + +_NONE_TYPE = type(None) +_PACKAGE = "je_auto_control" + + +# === Building a value from a JSON Schema node ================================ + +_SCALARS: Dict[str, Any] = { + "integer": 3, "number": 1.5, "boolean": True, + "array": [], "object": {}, "null": None, +} + +# The emptiest value each concrete annotation allows, as factories so two +# adapters never share one mutable container. + + +# A declared `format` narrows what "string" means, and an adapter that parses +# before delegating enforces it. Without this the sweep hands `ac_rrule_next` +# the generic sample and watches `datetime.fromisoformat` reject it -- which +# says nothing about the adapter and everything about the sample. +#: How far into a nested object the sample builder will follow `properties`. +_MAX_SCHEMA_DEPTH = 4 + +_STRING_FORMATS: Dict[str, str] = { + "date-time": "2026-01-02T03:04:05", + "date": "2026-01-02", + "time": "03:04:05", +} + + +def _sample_value(spec: Dict[str, Any], name: str, depth: int = 0) -> Any: + """Return a value satisfying one JSON-Schema property node. + + A declared ``object`` is built from its own ``properties`` rather than + left empty: a tool whose ``anchor`` argument names ``kind`` as required + is describing a payload no client would send as ``{}``, and handing the + adapter ``{}`` tests the sample, not the wiring. ``depth`` stops a schema + that describes itself. + """ + enum = spec.get("enum") + if isinstance(enum, list) and enum: + return enum[0] + kind = spec.get("type", "string") + if isinstance(kind, list): + kind = next((entry for entry in kind if entry != "null"), "string") + if kind == "array": + item = spec.get("items") + return [_sample_value(item, name, depth + 1)] if item else [] + if kind == "object": + properties = spec.get("properties") or {} + if not properties or depth >= _MAX_SCHEMA_DEPTH: + return {} + return {key: _sample_value(sub, key, depth + 1) + for key, sub in properties.items()} + if kind == "string": + return _STRING_FORMATS.get(spec.get("format"), f"value-for-{name}") + return _SCALARS.get(kind, f"value-for-{name}") + + +# === Reading a value out of a declared type ================================= + +_ZEROS: Dict[Any, Callable[[], Any]] = { + int: lambda: 0, float: lambda: 0.0, bool: lambda: False, + str: lambda: "", bytes: lambda: b"", + list: list, dict: dict, set: set, tuple: tuple, +} + +# An annotation is free to promise the abstract protocol rather than the +# concrete type, and `Mapping[str, X]` originates at `collections.abc.Mapping`, +# which cannot be instantiated. The emptiest value satisfying each is the +# builtin that registers as it. +_ABSTRACT_CONTAINERS: Dict[Any, Callable[[], Any]] = { + collections.abc.Mapping: dict, collections.abc.MutableMapping: dict, + collections.abc.Sequence: list, collections.abc.MutableSequence: list, + collections.abc.Iterable: list, collections.abc.Iterator: iter([]).__iter__, + collections.abc.Collection: list, + collections.abc.Set: set, collections.abc.MutableSet: set, +} + + + +def _value_for_generic(origin: Any, arguments: Tuple[Any, ...], + seen: FrozenSet[type] = frozenset()) -> Any: + """Build the emptiest value a parameterised annotation allows.""" + if origin is typing.Union: + return None if _NONE_TYPE in arguments else _value_for(arguments[0], + seen) + if origin is tuple: + if not arguments or arguments[-1] is Ellipsis: + return () + return tuple(_value_for(entry, seen) for entry in arguments) + if origin in (list, set, frozenset, dict): + return origin() + if origin in _ABSTRACT_CONTAINERS: + return _ABSTRACT_CONTAINERS[origin]() + raise ValueError(f"unmodelled container {origin!r}") + + +def _dataclass_value(cls: type, seen: FrozenSet[type]) -> Any: + """Build an instance of ``cls`` with every field at its emptiest value. + + A dataclass return annotation is not the end of the contract, it is one + more level of it: each field carries an annotation of its own, so "what + may the caller assume?" stays a question the program can answer. The + instance is real -- the adapter's ``.to_dict()``, its attribute reads and + its comparisons all run against the declared shape rather than a mock that + answers everything. + + ``seen`` breaks a cycle of dataclasses that reach each other by a bare + field; a self-reference through ``Optional`` or a container terminates on + its own, at None and at the empty container. + """ + if cls in seen: + raise ValueError(f"self-referential dataclass field on {cls!r}") + hints = typing.get_type_hints(cls) + arguments = {field.name: _value_for(hints.get(field.name, field.type), + seen | {cls}) + for field in dataclasses.fields(cls) if field.init} + return cls(**arguments) + + +def _constructed_value(cls: type, seen: FrozenSet[type]) -> Any: + """Build a real instance of a first-party class from its own ``__init__``. + + This is `_dataclass_value` one level along, and for the same reason: a + constructor's parameters carry annotations, so "what does calling this + promise the caller?" stays a question the program can answer. The instance + is the genuine class, not a double -- its methods run, its ``to_dict()`` + runs, and an adapter that calls one with the wrong arguments still raises. + That is what keeps "the adapter ran" and "the adapter was checked" from + coming apart, which a double answering every method would not. + + Only the constructor's *required* parameters are filled: a default is the + class's own statement of what the caller may leave out. + + Raising :class:`ValueError` leaves the adapter out of the sweep, which is + the right answer for all three ways this can fail -- a class outside the + package (the adapter is choosing a backend), a parameter the contract + never described, and zero values that break an invariant the annotation + cannot express (``rate must be positive`` for a token bucket, say). + """ + if cls in seen: + raise ValueError(f"self-referential constructor on {cls!r}") + if not getattr(cls, "__module__", "").startswith(_PACKAGE + "."): + raise ValueError(f"not a first-party class: {cls!r}") + if _module_picks_a_backend(cls.__module__): + raise ValueError(f"{cls!r} is defined beside a third-party import") + try: + signature = inspect.signature(cls.__init__) + hints = typing.get_type_hints(cls.__init__) + except (TypeError, ValueError, NameError) as error: + raise ValueError(f"unreadable constructor on {cls!r}: {error}") from error + arguments = {} + for name, parameter in signature.parameters.items(): + if name == "self" or parameter.default is not parameter.empty: + continue + if parameter.kind not in (parameter.POSITIONAL_OR_KEYWORD, + parameter.KEYWORD_ONLY): + continue + if name not in hints: + raise ValueError(f"{cls!r} takes an unannotated {name}") + arguments[name] = _value_for(hints[name], seen | {cls}) + try: + return cls(**arguments) + except Exception as error: # noqa: BLE001 # reason: any refusal means the contract's zero values do not build one, and the adapter leaves the sweep + raise ValueError( + f"{cls!r} refuses its contract's zero values: {error}") from error + + +@functools.lru_cache(maxsize=None) +def _module_picks_a_backend(module_path: str) -> bool: + """Return True when the module at ``module_path`` imports a third-party one. + + The sweeps already refuse an adapter that imports a third-party package, + because what it returns then depends on what is installed on the machine + rather than on any contract. A real instance moves that question one level + down: ``S3ArtifactStore`` constructs from its annotations perfectly well + and then reaches for ``boto3`` the moment a method is called. Same rule, + same seam, one level along -- and it keeps eight adapters off the + documented-exception list, where they would all have said "boto3". + """ + try: + source = inspect.getsource(sys.modules[module_path]) + except (OSError, TypeError, KeyError): + return True + for node in ast.walk(ast.parse(source)): + if isinstance(node, ast.Import): + if any(not _is_stdlib(alias.name) + and not alias.name.startswith(_PACKAGE) for alias in node.names): + return True + elif isinstance(node, ast.ImportFrom) and node.module and not node.level: + root = node.module.split(".")[0] + if not _is_stdlib(root) and root != _PACKAGE: + return True + return False + + +def _value_for(annotation: Any, seen: FrozenSet[type] = frozenset()) -> Any: + """Build the emptiest value a return annotation allows. + + ``Optional[X]`` yields None, containers yield empty containers, scalars + yield their zero, and a dataclass yields an instance built the same way + from its own fields. Raises :class:`ValueError` for an annotation this + cannot model, which leaves the adapter out of the sweep rather than + testing it against a value its callee never promised. + """ + if annotation is inspect.Signature.empty: + raise ValueError("no return annotation") + if annotation in (None, _NONE_TYPE, typing.Any): + return None + origin = typing.get_origin(annotation) + if origin is not None: + return _value_for_generic(origin, typing.get_args(annotation), seen) + if annotation in _ZEROS: + return _ZEROS[annotation]() + if isinstance(annotation, type) and dataclasses.is_dataclass(annotation): + return _dataclass_value(annotation, seen) + if isinstance(annotation, type): + return _constructed_value(annotation, seen) + raise ValueError(f"unmodelled return annotation {annotation!r}") + + +# === Finding the callee an adapter stands in front of ======================= + +def _adapter_source(adapter: Any) -> Optional[ast.FunctionDef]: + """Return the parsed definition of ``adapter``, or None if unavailable.""" + try: + source = textwrap.dedent(inspect.getsource(adapter)) + except (OSError, TypeError): + return None + node = ast.parse(source).body[0] + return node if isinstance(node, ast.FunctionDef) else None + + +def _is_stdlib(module_path: str) -> bool: + """Return True for a standard-library module path.""" + return module_path.split(".")[0] in sys.stdlib_module_names + + +def _is_foreign_import(node: ast.stmt) -> bool: + """Return True for an import that is neither stdlib nor first-party-from. + + A plain ``import`` of a third-party package means the adapter picks its own + backend; a relative import means the source could not be resolved to a + module path. Either way the adapter is out of scope. + """ + if isinstance(node, ast.Import): + return any(not _is_stdlib(alias.name) for alias in node.names) + return not node.module or bool(node.level) + + +def _first_party_from_imports(imports: List[ast.stmt] + ) -> List[Tuple[str, List[str]]]: + """Return ``(module, names)`` for each from-import inside this package. + + "First party" is the package prefix, not merely "not the stdlib": a + ``from PySide6... import`` is a backend choice like a plain third-party + import, and it is the prefix that lets the resolved path be imported below + without trusting whatever the parsed source happened to say. + """ + return [(child.module, [alias.name for alias in child.names]) + for child in imports + if isinstance(child, ast.ImportFrom) + and child.module.startswith(_PACKAGE + ".")] + + +def _project_import(adapter: Any) -> Optional[Tuple[str, List[str]]]: + """Return ``(module, names)`` for the one project callee an adapter imports. + + Standard-library imports are ignored: ``import json`` inside an adapter + parses a field the visual editor passed as text, and ``import base64`` + encodes what the callee returned -- neither is the callee. + """ + node = _adapter_source(adapter) + if node is None: + return None + imports = [child for child in ast.walk(node) + if isinstance(child, (ast.Import, ast.ImportFrom))] + if any(_is_foreign_import(child) for child in imports): + return None + found = _first_party_from_imports(imports) + return found[0] if len(found) == 1 else None + + +def _sole_imported_name(node: ast.stmt) -> Optional[Tuple[str, str, str]]: + """Return ``(module, attribute, local_name)`` for ``from X import y as z``. + + ``X`` has to be inside this package: a delegator standing in front of + ``json.dumps`` is not wiring under test, and it is the prefix that lets the + parsed path be imported below without trusting what the source said. + """ + if not isinstance(node, ast.ImportFrom) or not node.module or node.level: + return None + if len(node.names) != 1 or not node.module.startswith(_PACKAGE + "."): + return None + alias = node.names[0] + return node.module, alias.name, alias.asname or alias.name + + +def _returned_callee(node: ast.stmt) -> Optional[str]: + """Return the plain name a ``return name(...)`` statement calls.""" + if not isinstance(node, ast.Return): + return None + call = node.value + if not isinstance(call, ast.Call) or not isinstance(call.func, ast.Name): + return None + return call.func.id + + +def _delegated_call(node: ast.FunctionDef) -> Optional[Tuple[str, str]]: + """Return ``(module, attribute)`` for a body that is import-then-return.""" + body = [statement for statement in node.body + if not (isinstance(statement, ast.Expr) + and isinstance(statement.value, ast.Constant))] + if len(body) != 2: + return None + imported = _sole_imported_name(body[0]) + called = _returned_callee(body[1]) + if imported is None or called is None or called != imported[2]: + return None + return imported[0], imported[1] + + +def _delegation(adapter: Any) -> Optional[Tuple[str, str]]: + """Return ``(module, attribute)`` when ``adapter`` is a pure delegator. + + A pure delegator's body is exactly one ``from ... import name`` followed by + ``return name(...)``. Anything else -- a temporary, a branch, a second + import -- means the adapter does work of its own; the output sweeps cover + those instead. + """ + node = _adapter_source(adapter) + return None if node is None else _delegated_call(node) + + +def _sole_attribute_on(node: ast.FunctionDef, name: str) -> Optional[str]: + """Return the one attribute an adapter reaches for on the local ``name``. + + Several dozen adapters import a module-level singleton rather than a + function -- ``default_observer``, ``default_scheduler``, ``registry`` -- + and call one method on it. That is the same wiring shape one indirection + along, so the callee is the method and its annotation is the contract. + Two different methods mean the adapter orchestrates the object instead of + standing in front of one call, and it stays out of the sweep. + """ + attributes = {child.attr for child in ast.walk(node) + if isinstance(child, ast.Attribute) + and isinstance(child.value, ast.Name) + and child.value.id == name} + return attributes.pop() if len(attributes) == 1 else None + + +def _submodule(module: Any, name: str) -> Any: + """Return ``module.name`` when the imported name is a submodule. + + ``from ...usb.passthrough import commands`` binds a module, and a package + only grows that attribute once something has imported it -- which, for a + lazily-imported adapter, may not have happened yet. + """ + # Both halves are ``je_auto_control.*`` paths parsed out of this + # repository's own source; no caller supplies either. + path = f"{module.__name__}.{name}" + try: + return importlib.import_module(path) # nosemgrep # reason: prefix-checked at the parse site + except ImportError: + return None + + +def _callee_of(adapter: Any, module: Any, name: str + ) -> Optional[Tuple[Any, str]]: + """Return the ``(owner, attribute)`` pair an adapter actually calls.""" + target = getattr(module, name, None) + if callable(target): + return module, name + if target is None: + target = _submodule(module, name) + if target is None: + return None + node = _adapter_source(adapter) + attribute = None if node is None else _sole_attribute_on(node, name) + if attribute is None or not callable(getattr(target, attribute, None)): + return None + return target, attribute + + +def _contract_stubs(adapter: Any) -> Optional[List[Tuple[Any, str, Any]]]: + """Return ``(owner, attribute, value)`` for each of an adapter's callees. + + Each value comes from that callee's own return annotation, so the adapter + runs against exactly what the typing contract promises it. The owner is + the callee's module for a plain function and the singleton itself for a + method on one. + """ + found = _project_import(adapter) + if found is None: + return None + module_path, names = found + try: + # ``module_path`` is a ``je_auto_control.*`` path parsed out of this + # repository's own source; no caller supplies it. + module = importlib.import_module(module_path) # nosemgrep # reason: prefix-checked at the parse site + except ImportError: + return None + stubs: List[Tuple[Any, str, Any]] = [] + for name in names: + callee = _callee_of(adapter, module, name) + if callee is None: + return None + owner, attribute = callee + target = getattr(owner, attribute) + if isinstance(target, type): + # A class callee is left alone: the adapter constructs the real + # object out of the client's own arguments, which is both the + # strongest form of "run against the declared type" and the only + # one that keeps the arguments under test. Replacing the class + # with something returning a prepared instance would throw those + # arguments away -- and take the class's own constructors with + # them, since a stub standing in for a class has no `from_dict`. + # `_constructed_value` below still has to succeed, so a class the + # contract cannot build stays out of the sweep rather than being + # run blind. + try: + _value_for(target) + except (ValueError, TypeError, NameError): + return None + continue + try: + hints = typing.get_type_hints(target) + stubs.append((owner, attribute, + _value_for(hints.get("return", + inspect.Signature.empty)))) + except (ValueError, TypeError, NameError): + return None + return stubs + + +def _install_stubs(monkeypatch: pytest.MonkeyPatch, + stubs: List[Tuple[Any, str, Any]]) -> None: + """Replace each named callee with a stub returning its contract value.""" + for owner, attribute, value in stubs: + monkeypatch.setattr(owner, attribute, lambda *a, _v=value, **k: _v) + + +def _is_serialisable(value: Any, native: Tuple[type, ...] = ()) -> bool: + """Return True when a result can cross the JSON boundary. + + ``native`` names the types a registry encodes itself -- the MCP registry + turns its own content objects into JSON on the way out, so one of those, + or a non-empty list of them, is already at the boundary. + """ + if native and isinstance(value, native): + return True + if native and isinstance(value, list) and value and all( + isinstance(entry, native) for entry in value): + return True + try: + json.dumps(value) + except (TypeError, ValueError): + return False + return True + + +def _install_recorder(monkeypatch: pytest.MonkeyPatch, module_path: str, + attribute: str) -> Tuple[Dict[str, Any], Any, Callable]: + """Replace ``module_path.attribute`` with a call recorder. + + Returns the record dict, the sentinel the recorder returns, and the real + callable, whose signature says what the recorded arguments are named. + """ + # ``module_path`` is the ``je_auto_control.*`` path `_sole_imported_name` + # parsed and prefix-checked; no caller supplies it. + module = importlib.import_module(module_path) # nosemgrep # reason: prefix-checked at the parse site + original = getattr(module, attribute) + record: Dict[str, Any] = {} + sentinel = object() + + def _recorder(*args: Any, **kwargs: Any) -> Any: + record["args"] = args + record["kwargs"] = kwargs + return sentinel + + monkeypatch.setattr(module, attribute, _recorder) + return record, sentinel, original + + +# Public names for the sweeps; the underscored ones above stay private to the +# implementation so a future change here does not read as a change there. +value_for = _value_for +adapter_source = _adapter_source +project_import = _project_import +delegation = _delegation +contract_stubs = _contract_stubs +install_stubs = _install_stubs +sample_value = _sample_value +NONE_TYPE = _NONE_TYPE +install_recorder = _install_recorder +is_serialisable = _is_serialisable +is_stdlib = _is_stdlib diff --git a/test/unit_test/headless/_pyobjc_stub.py b/test/unit_test/headless/_pyobjc_stub.py new file mode 100644 index 00000000..b370ab52 --- /dev/null +++ b/test/unit_test/headless/_pyobjc_stub.py @@ -0,0 +1,231 @@ +"""A pyobjc stand-in for the macOS window backend, usable off macOS. + +`macos_backend.py` imports `Quartz`, `AppKit` and `ApplicationServices` +*inside* its methods, so the module loads on every platform and only the +calls need a Mac. Stubs in `sys.modules` therefore reach all of it from all +nine CI squares rather than the two Darwin ones -- which matters because the +coverage floor is the lowest square. + +Unlike the Xlib stub, the constants here are sentinels rather than real +values, and deliberately so: every one of them is either a dictionary key +into an info dict this file also creates, or an opaque token handed straight +back to a function this file also provides. Their numeric values never reach +any arithmetic in the code under test, so pinning them would pin nothing. +What *is* worth pinning is that the names exist at all -- +`test_pyobjc_stub_names.py` checks that against the installed frameworks on +the macOS squares. + +The accessibility half models one thing carefully: **AX functions return an +error code, where zero means success.** `close()` and `minimize()` return +`not ax.AXUIElementPerformAction(...)`, so a stub that returned a truthy +"success" would invert every one of those tests. + +Nothing here is a test; the file is named so pytest does not collect it. +""" +from __future__ import annotations + +import sys +import types + +#: Quartz's window-info dictionary keys, spelled as pyobjc spells them. +WINDOW_KEYS = ( + "kCGWindowNumber", "kCGWindowName", "kCGWindowLayer", + "kCGWindowOwnerPID", "kCGWindowBounds", +) + +#: The rest of the surface the backend names, by module. +QUARTZ_NAMES = WINDOW_KEYS + ( + "kCGWindowListOptionOnScreenOnly", "kCGWindowListExcludeDesktopElements", + "kCGNullWindowID", + "CGWindowListCopyWindowInfo", "CGPoint", "CGSize", +) +APPKIT_NAMES = ( + "NSWorkspace", "NSRunningApplication", + "NSApplicationActivateIgnoringOtherApps", +) +AX_NAMES = ( + "AXUIElementCreateApplication", "AXUIElementCopyAttributeValue", + "AXUIElementSetAttributeValue", "AXUIElementPerformAction", + "AXValueCreate", "AXValueGetValue", + # HIServices declares these and ApplicationServices inherits them; + # Quartz does not, which is what the macOS squares caught. + "kAXValueCGPointType", "kAXValueCGSizeType", +) + +#: What an AX call returns when it worked. Zero, and the backend reads it as +#: `not error`, so this is load-bearing in every action test. +AX_SUCCESS = 0 +AX_FAILURE = -25200 # kAXErrorCannotComplete, in spirit + + +def window_info(number: int, *, name: str = "", layer: int = 0, + pid: int = 0, bounds=None) -> dict: + """One entry of what `CGWindowListCopyWindowInfo` returns.""" + info = { + "kCGWindowNumber": number, + "kCGWindowName": name, + "kCGWindowLayer": layer, + "kCGWindowOwnerPID": pid, + } + if bounds is not None: + left, top, width, height = bounds + info["kCGWindowBounds"] = {"X": left, "Y": top, + "Width": width, "Height": height} + return info + + +class AXElement: + """An accessibility element: attributes, and what was done to it.""" + + def __init__(self, **attributes) -> None: + self.attributes = dict(attributes) + self.actions = [] + self.assignments = [] + self.set_error = AX_SUCCESS + self.action_error = AX_SUCCESS + self.read_error = AX_SUCCESS + + +class AXPoint: + """What `AXValueCreate(kAXValueCGPointType, ...)` hands back.""" + + def __init__(self, x, y) -> None: + self.x = x + self.y = y + + +class AXSize: + def __init__(self, width, height) -> None: + self.width = width + self.height = height + + +class World: + """The Mac the backend thinks it is talking to.""" + + def __init__(self, windows=None, ax_windows=None, frontmost_pid=None, + running_pids=None): + self.windows = list(windows or []) + self.ax_windows = dict(ax_windows or {}) # pid -> [AXElement] + self.frontmost_pid = frontmost_pid + # Whether `NSRunningApplication` still answers for a pid. Separate + # from `ax_windows` on purpose: an application can quit between the + # pid lookup and the activation, which is its own branch. + self.running_pids = (set() if running_pids is None + else set(running_pids)) + self.list_options = [] + self.activated = [] + self.ax_list_error = AX_SUCCESS + + # -- Quartz -- + def copy_window_info(self, options, relative_to): + self.list_options.append((options, relative_to)) + return list(self.windows) + + # -- ApplicationServices -- + def ax_application(self, pid): + return ("application", int(pid)) + + def ax_copy_attribute(self, element, attribute, _placeholder): + if isinstance(element, tuple) and element[0] == "application": + if attribute == "AXWindows": + if self.ax_list_error: + return (self.ax_list_error, None) + return (AX_SUCCESS, self.ax_windows.get(element[1], [])) + return (AX_FAILURE, None) + if element.read_error: + return (element.read_error, None) + if attribute not in element.attributes: + return (AX_FAILURE, None) + return (AX_SUCCESS, element.attributes[attribute]) + + def ax_set_attribute(self, element, attribute, value): + element.assignments.append((attribute, value)) + if element.set_error: + return element.set_error + element.attributes[attribute] = value + return AX_SUCCESS + + def ax_perform_action(self, element, action): + element.actions.append(action) + return element.action_error + + def ax_value_create(self, kind, value): + return (kind, value) + + def ax_value_get(self, value, kind, _placeholder): + if not isinstance(value, tuple) or value[0] != kind: + return (False, None) + return (True, value[1]) + + +def install(monkeypatch, world: World) -> World: + """Put Quartz, AppKit and ApplicationServices in `sys.modules`.""" + quartz = types.ModuleType("Quartz") + for name in WINDOW_KEYS: + setattr(quartz, name, name) + quartz.kCGWindowListOptionOnScreenOnly = 1 + quartz.kCGWindowListExcludeDesktopElements = 16 + quartz.kCGNullWindowID = 0 + quartz.CGWindowListCopyWindowInfo = world.copy_window_info + quartz.CGPoint = AXPoint + quartz.CGSize = AXSize + + appkit = types.ModuleType("AppKit") + appkit.NSApplicationActivateIgnoringOtherApps = 2 + appkit.NSWorkspace = _Workspace(world) + appkit.NSRunningApplication = _RunningApplication(world) + + services = types.ModuleType("ApplicationServices") + services.AXUIElementCreateApplication = world.ax_application + services.AXUIElementCopyAttributeValue = world.ax_copy_attribute + services.AXUIElementSetAttributeValue = world.ax_set_attribute + services.AXUIElementPerformAction = world.ax_perform_action + services.AXValueCreate = world.ax_value_create + services.AXValueGetValue = world.ax_value_get + services.kAXValueCGPointType = "point" + services.kAXValueCGSizeType = "size" + + for name, module in (("Quartz", quartz), ("AppKit", appkit), + ("ApplicationServices", services)): + monkeypatch.setitem(sys.modules, name, module) + return world + + +def install_missing(monkeypatch, name: str = "Quartz") -> None: + """Make one framework import fail, as a Mac without pyobjc does. + + `None` in `sys.modules` is the import system's own way of recording + "this one is not there": `import Quartz` then raises ImportError without + going near the filesystem, which is exactly what the probe is written to + survive. + """ + monkeypatch.setitem(sys.modules, name, None) + + +class _Workspace: + def __init__(self, world: World) -> None: + self._world = world + + def sharedWorkspace(self): # noqa: N802 # reason: the AppKit name + return self + + def frontmostApplication(self): # noqa: N802 # reason: the AppKit name + if self._world.frontmost_pid is None: + return None + return types.SimpleNamespace( + processIdentifier=lambda: self._world.frontmost_pid) + + +class _RunningApplication: + def __init__(self, world: World) -> None: + self._world = world + + def runningApplicationWithProcessIdentifier_(self, pid): # noqa: N802 + if pid not in self._world.running_pids: + return None + world = self._world + + return types.SimpleNamespace( + activateWithOptions_=lambda options: world.activated.append( + (pid, options))) diff --git a/test/unit_test/headless/_uia_doubles.py b/test/unit_test/headless/_uia_doubles.py new file mode 100644 index 00000000..dcc79da5 --- /dev/null +++ b/test/unit_test/headless/_uia_doubles.py @@ -0,0 +1,246 @@ +"""A UIAutomation stand-in, so the Windows accessibility backend is testable. + +`backends/windows_backend.py` is the largest single gap in the project's +coverage -- 568 statements at 17% -- and it stayed there on the Windows +squares too, because reaching it needs a UIAutomation provider, a desktop +with windows on it, and applications willing to answer. None of that exists +on a CI runner of any platform. + +It does not need any of it. Every `comtypes` import in the module is inside a +function, the automation object is held on the instance, and each control +pattern is reached the same way: find a raw element, ask it for a pattern id, +query an interface off the generated module. All three are values a test can +supply. + +The shape modelled here is that indirection, because it is where a mistake +hides: + + unknown = raw.GetCurrentPattern(pattern_id) # None if unsupported + interface = getattr(uia_module, interface_name) # generated at import + pattern = unknown.QueryInterface(interface) + +A control that does not support a pattern answers with nothing, and every +caller has to turn that into its own "no" -- `None`, `False`, or `[]` +depending on what it promised to return. `Unknown` below therefore refuses to +hand back a pattern for the wrong interface name, so a method that asks for +`IUIAutomationValuePattern` where it meant `IUIAutomationRangeValuePattern` +fails here rather than silently working against a double that answers +everything. + +Nothing here is a test; the file is named so pytest does not collect it. +""" +from __future__ import annotations + +import sys +import types + + +class Rect: + """What UIA's BoundingRectangle property hands back.""" + + def __init__(self, left=0, top=0, right=0, bottom=0) -> None: + self.left = left + self.top = top + self.right = right + self.bottom = bottom + + +class Pattern: + """One control pattern: attributes to read, and calls to record.""" + + def __init__(self, **attributes) -> None: + self.__dict__.update(attributes) + self.calls = [] + self.errors = {} # method name -> exception to raise + + def _record(self, name, *args): + self.calls.append((name, args)) + if name in self.errors: + raise self.errors[name] + return None + + def __getattr__(self, name): + if name.startswith("_"): + raise AttributeError(name) + # A UIA *property* is spelled `Current…` / `Cached…`; one that was + # not supplied is one the provider will not answer, which reaches the + # backend as AttributeError and is a path it handles. Answering with + # a callable instead would let `str(pattern.CurrentValue or "")` pass + # against the repr of a function. + if name.startswith(("Current", "Cached")): + raise AttributeError(name) + + def _call(*args): + return self._record(name, *args) + return _call + + +class Unknown: + """What `GetCurrentPattern` returns: an IUnknown to query an interface off.""" + + def __init__(self, interface_name: str, pattern: Pattern) -> None: + self.interface_name = interface_name + self.pattern = pattern + self.queried = [] + + def QueryInterface(self, interface): # noqa: N802 # reason: COM name + self.queried.append(interface) + if interface != self.interface_name: + raise TypeError( + f"asked for {interface!r}, this pattern is " + f"{self.interface_name!r}") + return self.pattern + + +class RawElement: + """A raw UIA element: properties, patterns, and what was done to it.""" + + def __init__(self, name="", control_type=0, rect=None, process_id=0, + automation_id="", enabled=True, patterns=None, + properties=None, cached=False) -> None: + self.patterns = dict(patterns or {}) # pattern id -> Unknown + self.properties = dict(properties or {}) # property id -> value + self.focused = 0 + self.pattern_error = None + self.property_error = None + self.focus_error = None + prefix = "Cached" if cached else "Current" + setattr(self, prefix + "Name", name) + setattr(self, prefix + "ControlType", control_type) + setattr(self, prefix + "BoundingRectangle", rect or Rect()) + setattr(self, prefix + "ProcessId", process_id) + setattr(self, prefix + "AutomationId", automation_id) + setattr(self, prefix + "IsEnabled", enabled) + # `_convert_uia` reads one prefix or the other; a raw element that + # only ever appears cached must not answer to `Current*`. + self._prefix = prefix + + def GetCurrentPattern(self, pattern_id): # noqa: N802 # reason: UIA name + if self.pattern_error is not None: + raise self.pattern_error + return self.patterns.get(pattern_id) + + def GetCurrentPropertyValue(self, property_id): # noqa: N802 # UIA name + if self.property_error is not None: + raise self.property_error + return self.properties.get(property_id) + + def SetFocus(self): # noqa: N802 # reason: the UIA name + if self.focus_error is not None: + raise self.focus_error + self.focused += 1 + + +class ElementArray: + """An `IUIAutomationElementArray`: a length and indexed access.""" + + def __init__(self, elements=None, length_error=None, + element_error=None) -> None: + self.elements = list(elements or []) + self.length_error = length_error + self.element_error = element_error + + @property + def Length(self): # noqa: N802 # reason: the UIA name + if self.length_error is not None: + raise self.length_error + return len(self.elements) + + def GetElement(self, index): # noqa: N802 # reason: the UIA name + if self.element_error is not None: + raise self.element_error + return self.elements[index] + + +class UiaModule: + """The comtypes-generated `UIAutomationClient` module. + + Its interfaces are generated at import time, so a name is all there is to + hold on to; `getattr` therefore answers with the name itself and + `Unknown` compares against it. + """ + + def __init__(self, has_iuiautomation2: bool = True) -> None: + self._has_2 = has_iuiautomation2 + self.IUIAutomation = "IUIAutomation" + + def __getattr__(self, name): + # An older Windows generates a module without IUIAutomation2 at all, + # which is the fallback the backend is written to take -- so the + # absence has to be a real AttributeError, not a name that answers. + if name.startswith("_"): + raise AttributeError(name) + if name == "IUIAutomation2" and not self._has_2: + raise AttributeError(name) + return name + + +class Automation: + """The UIAutomation object: roots, a walker, and event handler bookkeeping.""" + + def __init__(self, root=None) -> None: + self.root = root + self.focus_handlers = [] + self.add_error = None + self.remove_error = None + self.ConnectionTimeout = None # noqa: N815 # reason: the UIA name + + def GetRootElement(self): # noqa: N802 # reason: the UIA name + return self.root + + def AddFocusChangedEventHandler(self, cache_request, handler): # noqa: N802 + if self.add_error is not None: + raise self.add_error + self.focus_handlers.append(handler) + + def RemoveFocusChangedEventHandler(self, handler): # noqa: N802 + if self.remove_error is not None: + raise self.remove_error + if handler in self.focus_handlers: + self.focus_handlers.remove(handler) + + +def with_pattern(pattern_id: int, interface_name: str, + pattern: Pattern = None) -> dict: + """A `patterns` mapping carrying one pattern, ready for `RawElement`.""" + return {pattern_id: Unknown(interface_name, pattern or Pattern())} + + +def install_comtypes(monkeypatch, uia_module=None, module_error=None, + create=None): + """Put a `comtypes` package in `sys.modules` and return what it hands out. + + Only the four names the backend reaches for: `client.GetModule`, which + generates the UIAutomationClient module, and `CoCreateInstance` / `GUID` / + `COMObject`, which build the automation object and the event handler. + """ + created = [] + + def _co_create_instance(clsid, interface=None): + created.append((clsid, interface)) + if create is not None: + return create(clsid, interface) + return Automation() + + comtypes = types.ModuleType("comtypes") + comtypes.CoCreateInstance = _co_create_instance + comtypes.GUID = lambda text: text + + class _ComObject: + """The base comtypes gives a Python-implemented COM interface.""" + + comtypes.COMObject = _ComObject + + client = types.ModuleType("comtypes.client") + + def _get_module(name): + if module_error is not None: + raise module_error + return uia_module if uia_module is not None else UiaModule() + + client.GetModule = _get_module + comtypes.client = client + + monkeypatch.setitem(sys.modules, "comtypes", comtypes) + monkeypatch.setitem(sys.modules, "comtypes.client", client) + return created diff --git a/test/unit_test/headless/_webrtc_doubles.py b/test/unit_test/headless/_webrtc_doubles.py new file mode 100644 index 00000000..eb663b7d --- /dev/null +++ b/test/unit_test/headless/_webrtc_doubles.py @@ -0,0 +1,307 @@ +"""The aiortc-shaped doubles the WebRTC tests share. + +Six test modules -- host session, host channels, host inbox, viewer session, +viewer media, viewer control -- all stand the same two classes up against the +same three collaborators, and every one of those collaborators is a boundary +we deliberately do not cross in a unit test: + +* **`RTCPeerConnection`** would start ICE traffic against public STUN + servers from a CI runner. +* **`ScreenVideoTrack`** would open a screen grabber, which on a headless + runner is either absent or a black rectangle. +* **The asyncio bridge** would queue work onto a background event loop that + no test is running, so nothing would ever be observed. + +What is faked is only the aiortc surface: the doubles record what they were +handed and hand back what the real objects hand back. The code under test -- +which transceiver the host adds, which slot the viewer replaces, which +envelope goes out on which channel -- is never replaced. + +`FakePeerConnection` carries both ends' surface in one class on purpose. It +stands in for one aiortc type, and splitting it per direction would mean two +doubles drifting apart from the same original. + +Nothing here is a test; the file is named so pytest does not collect it. +""" +from __future__ import annotations + +import asyncio +from concurrent.futures import Future + + +class Channel: + """A DataChannel: records its handlers, and what was sent on it.""" + + def __init__(self, label: str = "ctrl", + ready_state: str = "connecting") -> None: + self.label = label + self.readyState = ready_state # noqa: N815 # reason: the aiortc name + self.handlers = {} + self.sent = [] + self.send_error = None + + def on(self, event): + def _register(func): + self.handlers[event] = func + return func + return _register + + def fire(self, event, *args): + return self.handlers[event](*args) + + def send(self, payload): + if self.send_error is not None: + raise self.send_error + self.sent.append(payload) + + +class Track: + """A MediaStreamTrack: a kind, and whether anybody stopped it.""" + + def __init__(self, kind: str = "video", **kwargs) -> None: + self.kind = kind + self.kwargs = kwargs + self.stopped = False + + def stop(self) -> None: + self.stopped = True + + +class FrameTrack: + """Yields a scripted sequence of frames, then raises to end the stream.""" + + def __init__(self, *frames, ending=None) -> None: + from aiortc.mediastreams import MediaStreamError + self.frames = list(frames) + self._ending = ending if ending is not None else MediaStreamError() + + async def recv(self): + if not self.frames: + raise self._ending + return self.frames.pop(0) + + +class Sender: + def __init__(self, track=None) -> None: + self.track = track + self.replaced = [] + self.replace_error = None + + def replaceTrack(self, track): # noqa: N802 # reason: the aiortc name + if self.replace_error is not None: + raise self.replace_error + self.replaced.append(track) + self.track = track + + +class Transceiver: + def __init__(self, kind: str, track=None, + direction: str = "recvonly") -> None: + self.kind = kind + self.sender = Sender(track) + self.direction = direction + + +class FakePeerConnection: + """The aiortc surface both ends touch, and nothing beyond it.""" + + instances: list = [] + + def __init__(self, configuration=None) -> None: + self.configuration = configuration + self.connectionState = "new" # noqa: N815 # reason: the aiortc name + self.iceGatheringState = "new" # noqa: N815 # reason: the aiortc name + self.tracks = [] + self.transceivers = [] + self.channels = [] + self.handlers = {} + self.local_sdp = "v=0 local-sdp" + self.remote_descriptions = [] + self.closed = False + self.stats = {} + self.stats_error = None + self.answer_error = None + self.remote_description_error = None + FakePeerConnection.instances.append(self) + + # --- the aiortc surface -------------------------------------------------- + + def addTrack(self, track): # noqa: N802 # reason: the aiortc name + self.tracks.append(track) + + def addTransceiver(self, kind, direction): # noqa: N802 # the aiortc name + self.transceivers.append((kind, direction)) + + def getTransceivers(self): # noqa: N802 # reason: the aiortc name + return list(self.transceivers) + + def createDataChannel(self, label): # noqa: N802 # reason: the aiortc name + channel = Channel(label) + self.channels.append(channel) + return channel + + def on(self, event): + def _register(func): + self.handlers[event] = func + return func + return _register + + async def createOffer(self): # noqa: N802 # reason: the aiortc name + return "offer-object" + + async def createAnswer(self): # noqa: N802 # reason: the aiortc name + if self.answer_error is not None: + raise self.answer_error + return "answer-object" + + async def setLocalDescription(self, description): # noqa: N802 + self.local_description_arg = description + + async def setRemoteDescription(self, description): # noqa: N802 + if self.remote_description_error is not None: + raise self.remote_description_error + self.remote_descriptions.append(description) + + async def getStats(self): # noqa: N802 # reason: the aiortc name + if self.stats_error is not None: + raise self.stats_error + return self.stats + + async def close(self): + self.closed = True + + @property + def localDescription(self): # noqa: N802 # reason: the aiortc name + return type("_Desc", (), {"sdp": self.local_sdp})() + + # --- test helpers -------------------------------------------------------- + + def channel(self, label: str) -> Channel: + return next(c for c in self.channels if c.label == label) + + def fire(self, event, *args): + return self.handlers[event](*args) + + def video_slots(self, count: int = 2): + """Offer `count` video transceivers, as a host's offer would.""" + self.transceivers = [Transceiver("video") for _ in range(count)] + return self.transceivers + + def complete_ice_gathering(self) -> None: + """Finish gathering and notify whoever subscribed to the change.""" + self.iceGatheringState = "complete" + self.handlers["icegatheringstatechange"]() + + +class Bridge: + """The asyncio bridge, minus the loop: run the work here and now. + + `submit` mirrors `run_coroutine_threadsafe` in parking any exception on + the future rather than raising into the caller, because that is where + `create_offer`'s and `stop`'s error handling reads it from. + """ + + def __init__(self) -> None: + self.deferred = [] + + def submit(self, coro) -> Future: + future: Future = Future() + try: + future.set_result(asyncio.run(coro)) + except BaseException as error: # noqa: BLE001 # reason: mirrors run_coroutine_threadsafe, which parks any exception on the future rather than raising into the submitting thread + future.set_exception(error) + return future + + def call_soon(self, callback, *args) -> None: + self.deferred.append((callback, args)) + callback(*args) + + +class HangingBridge: + """A bridge whose work never lands: every future times out.""" + + def submit(self, coro) -> Future: + coro.close() + future: Future = Future() + future.set_exception(asyncio.TimeoutError()) + return future + + def call_soon(self, callback, *args) -> None: + return None + + +class AuditLog: + """The audit log, without the write to the user's real home directory.""" + + def __init__(self) -> None: + self.events = [] + self.error = None + + def log(self, event_type, **fields) -> None: + if self.error is not None: + raise self.error + self.events.append((event_type, fields)) + + @property + def event_types(self) -> list: + return [event for event, _ in self.events] + + +class MicReceiver: + """Host side of the raw-PCM mic uplink: what arrived, and was it closed.""" + + def __init__(self) -> None: + self.chunks = [] + self.started = False + self.stopped = False + self.stop_error = None + + def start(self) -> None: + self.started = True + + def on_chunk(self, chunk) -> None: + self.chunks.append(chunk) + + def stop(self) -> None: + if self.stop_error is not None: + raise self.stop_error + self.stopped = True + + +class MicSender: + """Viewer side of the same uplink; it is handed the channel to write to.""" + + def __init__(self, channel) -> None: + self.channel = channel + self.started = False + self.running = True + self.stop_error = None + + def start(self) -> None: + self.started = True + + def is_running(self) -> bool: + return self.running + + def stop(self) -> None: + if self.stop_error is not None: + raise self.stop_error + self.running = False + + +class Stoppable: + """Something the teardown path is expected to stop, and may fail to.""" + + def __init__(self, error=None) -> None: + self.stopped = False + self._error = error + + def stop(self) -> None: + if self._error is not None: + raise self._error + self.stopped = True + + +async def noop_ice_gathering(pc, timeout=None): + """Stand in for `wait_for_ice_gathering`: there are no candidates here.""" + return None diff --git a/test/unit_test/headless/_xlib_stub.py b/test/unit_test/headless/_xlib_stub.py new file mode 100644 index 00000000..a6fce2d1 --- /dev/null +++ b/test/unit_test/headless/_xlib_stub.py @@ -0,0 +1,319 @@ +"""A python-Xlib stand-in, so the X11 backends can be tested off Linux. + +`python-Xlib` is a Linux/BSD-only dependency, and both X11 backends import it +*inside* their methods — so the modules load anywhere and only the calls need +a display. That is what makes them testable on all nine CI squares rather +than the two that have X: put a stub in `sys.modules` and the lazy import +finds it, on Linux as well as on Windows and macOS. + +Two things make this a stub rather than a mock. + +**The constants carry their real values.** `SubstructureRedirectMask` here is +`1 << 20` because that is what `X.h` says, so a test asserting an event mask +is asserting the number that goes on the wire. `test_xlib_stub_values.py` +compares every constant below against the real module wherever it is +installed, which on CI is both Linux squares — so a wrong value here is a +named failure there rather than a test that agrees with itself. + +**The display is a real object graph, not a recorder.** Windows have +properties, properties are keyed by interned atom, and `query_tree` returns a +parent — because the code under test walks up to the frame, interns atoms +once per connection, and reads properties by id. A recorder would let a +backend that never interned an atom pass. + +Nothing here is a test; the file is named so pytest does not collect it. +""" +from __future__ import annotations + +import sys +import types + +# --- X.h ----------------------------------------------------------------- + +#: `X.h` event masks, modifier masks and sentinels the backends name. +X_CONSTANTS = { + "NONE": 0, + "CurrentTime": 0, + "AnyPropertyType": 0, + "KeyPress": 2, + "KeyPressMask": 1 << 0, + "KeyReleaseMask": 1 << 1, + "ButtonPressMask": 1 << 2, + "ButtonReleaseMask": 1 << 3, + "SubstructureNotifyMask": 1 << 19, + "SubstructureRedirectMask": 1 << 20, + # Modifier masks, for the hotkey backend's grabs. The two lock masks are + # what it re-grabs each combo under, so a hotkey still fires with NumLock + # or CapsLock on. + "ShiftMask": 1 << 0, + "LockMask": 1 << 1, + "ControlMask": 1 << 2, + "Mod1Mask": 1 << 3, + "Mod2Mask": 1 << 4, + "Mod4Mask": 1 << 6, + "GrabModeSync": 0, + "GrabModeAsync": 1, +} + +#: Keysyms `XK.string_to_keysym` answers for, by their `keysymdef.h` values. +#: Latin-1 letters are their ASCII code point; the named keys are 0xFF00-range +#: function keysyms. +KEYSYMS = { + "a": 0x0061, "b": 0x0062, "k": 0x006B, "q": 0x0071, "z": 0x007A, + "1": 0x0031, "Return": 0xFF0D, "Tab": 0xFF09, "Escape": 0xFF1B, + "space": 0x0020, "F5": 0xFFC2, "Page_Up": 0xFF55, "Delete": 0xFFFF, +} + +#: `Xatom.h` predefined atoms, by their fixed protocol numbers. +XATOM_CONSTANTS = { + "CARDINAL": 6, + "STRING": 31, + "WINDOW": 33, +} + + +# --- the object graph ---------------------------------------------------- + +class Event: + """One X event, recording the keywords the backend built it from.""" + + def __init__(self, kind: str, **fields) -> None: + self.kind = kind + self.fields = fields + + def __repr__(self) -> str: # pragma: no cover - debugging aid + return f"Event({self.kind!r}, {self.fields!r})" + + +class _Property: + def __init__(self, value) -> None: + self.value = value + + +class Window: + """A window with properties, a parent, and a geometry.""" + + def __init__(self, display, window_id: int, *, properties=None, + geometry=None, wm_name=None, parent_id=None) -> None: + self._display = display + self.id = int(window_id) + self.properties = dict(properties or {}) + self.geometry = geometry or (0, 0, 0, 0) + self.wm_name = wm_name + self.parent_id = parent_id + self.sent = [] + self.mapped = None + self.attributes = {} + self.grabs = [] + self.ungrabs = [] + self.grab_errors = {} # (keycode, mask) -> exception + self.ungrab_errors = {} + self.property_error = None + self.tree_error = None + self.geometry_error = None + self.send_error = None + + # -- the python-Xlib surface -- + def get_full_property(self, atom: int, kind): + if self.property_error is not None: + raise self.property_error + name = self._display.atom_name(atom) + if name not in self.properties: + return None + return _Property(self.properties[name]) + + def get_wm_name(self): + return self.wm_name + + def query_tree(self): + if self.tree_error is not None: + raise self.tree_error + parent = (None if self.parent_id is None + else self._display.window(self.parent_id)) + return types.SimpleNamespace(parent=parent) + + def get_geometry(self): + if self.geometry_error is not None: + raise self.geometry_error + x, y, width, height = self.geometry + return types.SimpleNamespace(x=x, y=y, width=width, height=height) + + def send_event(self, event, event_mask=0, propagate=False): + if self.send_error is not None: + raise self.send_error + self.sent.append((event, event_mask, propagate)) + + def map(self): + self.mapped = True + + def unmap(self): + self.mapped = False + + def change_attributes(self, **kwargs): + self.attributes = dict(kwargs) + + def grab_key(self, keycode, mask, owner_events, pointer_mode, key_mode): + error = self.grab_errors.get((keycode, mask)) + if error is not None: + raise error + self.grabs.append((keycode, mask, owner_events, pointer_mode, + key_mode)) + + def ungrab_key(self, keycode, mask): + if (keycode, mask) in self.ungrab_errors: + raise self.ungrab_errors[(keycode, mask)] + self.ungrabs.append((keycode, mask)) + + @property + def grabbed(self): + """The `(keycode, mask)` pairs currently held, in grab order.""" + held = [(keycode, mask) for keycode, mask, *_rest in self.grabs] + for pair in self.ungrabs: + if pair in held: + held.remove(pair) + return held + + +class Display: + """One X connection: an atom table, a root, and a window table.""" + + #: Interned atoms start above the predefined range so a test can tell an + #: interned atom from `Xatom.WINDOW` at a glance. + FIRST_INTERNED_ATOM = 100 + + def __init__(self) -> None: + self.atoms = {} + self.flushes = 0 + self.syncs = 0 + self.closed = False + self.events = [] + self.keycodes = {} # keysym -> keycode + self._windows = {} + self.root = Window(self, 1) + self._windows[1] = self.root + + # -- the python-Xlib surface -- + def intern_atom(self, name: str) -> int: + if name not in self.atoms: + self.atoms[name] = self.FIRST_INTERNED_ATOM + len(self.atoms) + return self.atoms[name] + + def screen(self): + return types.SimpleNamespace(root=self.root) + + def create_resource_object(self, kind: str, window_id: int) -> Window: + assert kind == "window" + return self.window(window_id) + + def flush(self) -> None: + self.flushes += 1 + + def sync(self) -> None: + self.syncs += 1 + + def close(self) -> None: + self.closed = True + + def keysym_to_keycode(self, keysym: int) -> int: + return self.keycodes.get(keysym, 0) + + def pending_events(self) -> int: + return len(self.events) + + def next_event(self): + return self.events.pop(0) + + # -- test helpers -- + def atom_name(self, atom: int): + for name, value in self.atoms.items(): + if value == atom: + return name + return None + + def window(self, window_id: int, **kwargs) -> Window: + window_id = int(window_id) + if window_id not in self._windows: + self._windows[window_id] = Window(self, window_id, **kwargs) + elif kwargs: + existing = self._windows[window_id] + for key, value in kwargs.items(): + setattr(existing, key, value) + return self._windows[window_id] + + def set_root_property(self, name: str, value) -> None: + """Publish an EWMH property on the root, interning its atom.""" + self.intern_atom(name) + self.root.properties[name] = value + + +def install(monkeypatch, display=None, display_error=None) -> Display: + """Put the stub in `sys.modules` and hand back the display it opens. + + Shadows the real `Xlib` for the duration wherever one is installed, so + the same test reads the same on every square. + + `display_error` makes `Display()` raise while leaving the rest of the + package importable, which is the shape of a Wayland or headless session: + python-Xlib is a hard dependency on Linux, so it is always *there* — it + is the connection that is not. + """ + display = display if display is not None else Display() + + x_module = types.ModuleType("Xlib.X") + for name, value in X_CONSTANTS.items(): + setattr(x_module, name, value) + + xatom_module = types.ModuleType("Xlib.Xatom") + for name, value in XATOM_CONSTANTS.items(): + setattr(xatom_module, name, value) + + # `XK.string_to_keysym` answers 0 for a name X does not know, which is + # the "unknown key" branch the hotkey backend has to report. + xk_module = types.ModuleType("Xlib.XK") + xk_module.string_to_keysym = lambda name: KEYSYMS.get(name, 0) + + event_module = types.SimpleNamespace( + ClientMessage=lambda **kwargs: Event("ClientMessage", **kwargs), + KeyPress=lambda **kwargs: Event("KeyPress", **kwargs), + KeyRelease=lambda **kwargs: Event("KeyRelease", **kwargs), + ButtonPress=lambda **kwargs: Event("ButtonPress", **kwargs), + ButtonRelease=lambda **kwargs: Event("ButtonRelease", **kwargs), + ) + protocol_module = types.ModuleType("Xlib.protocol") + protocol_module.event = event_module + + def _open_display(*args, **kwargs): + if display_error is not None: + raise display_error + return display + + display_module = types.ModuleType("Xlib.display") + display_module.Display = _open_display + + package = types.ModuleType("Xlib") + package.X = x_module + package.Xatom = xatom_module + package.XK = xk_module + package.protocol = protocol_module + package.display = display_module + + for name, module in (("Xlib", package), ("Xlib.X", x_module), + ("Xlib.Xatom", xatom_module), + ("Xlib.XK", xk_module), + ("Xlib.protocol", protocol_module), + ("Xlib.display", display_module)): + monkeypatch.setitem(sys.modules, name, module) + return display + + +def install_failing(monkeypatch, error) -> None: + """Install a stub whose `Display()` raises, as a headless session does.""" + def _raise(*args, **kwargs): + raise error + + display_module = types.ModuleType("Xlib.display") + display_module.Display = _raise + package = types.ModuleType("Xlib") + package.display = display_module + monkeypatch.setitem(sys.modules, "Xlib", package) + monkeypatch.setitem(sys.modules, "Xlib.display", display_module) diff --git a/test/unit_test/headless/test_accessibility_backend_selection.py b/test/unit_test/headless/test_accessibility_backend_selection.py new file mode 100644 index 00000000..fe9584e9 --- /dev/null +++ b/test/unit_test/headless/test_accessibility_backend_selection.py @@ -0,0 +1,216 @@ +"""Which accessibility backend a platform gets, and what the rest refuse. + +The seam is the same shape as the window-management one -- abstract base, +three implementations, a null fallback carrying a reason -- and it had the +same hole: the selector only ever ran its own platform's arm, so two of the +three branches were dead on every square, along with the base class's whole +refusal surface. + +Two things are worth stating in a test rather than in a comment: + +* **The reason has to name the *right* missing thing.** These are the + messages an operator reads when `ac_list_accessibility_elements` comes back + empty, and each platform fails for a different reason: a missing pip + package on Windows, a missing framework on macOS, and on Linux a *bus* that + may be absent or merely unbridged -- which is why that one names + at-spi2-core and the toolkit bridge in the same breath. +* **The base refuses rather than answering falsely.** Thirty-odd control + patterns exist because UIA has them; AT-SPI and AX have a fraction. A + backend that returned `None` or `False` for the rest would be indis- + tinguishable from "the control is not there", and a caller cannot recover + from that. Every one of them raises instead. + +The macOS backend is exercised here too, through the pyobjc stub, which is +what lets all of this run on every square rather than only on Darwin. +""" +from __future__ import annotations + +import sys +import types + +import pytest + +from headless import _pyobjc_stub as objc_stub +from je_auto_control.utils.accessibility import backends as backends_mod +from je_auto_control.utils.accessibility.backends import ( + NullAccessibilityBackend, get_backend, reset_backend_cache, +) +from je_auto_control.utils.accessibility.backends.base import ( + AccessibilityBackend, +) +from je_auto_control.utils.accessibility.element import ( + AccessibilityNotAvailableError, +) + + +@pytest.fixture(autouse=True) +def clean_cache(): + reset_backend_cache() + yield + reset_backend_cache() + + +@pytest.fixture +def on_platform(monkeypatch): + def _set(name: str): + monkeypatch.setattr(backends_mod.sys, "platform", name) + return _set + + +def _stub_module(monkeypatch, name: str, **attributes): + module = types.ModuleType(name) + for key, value in attributes.items(): + setattr(module, key, value) + monkeypatch.setitem(sys.modules, name, module) + return module + + +# --- selection ---------------------------------------------------------------- + +def test_windows_with_comtypes_selects_the_uia_backend(on_platform, + monkeypatch): + on_platform("win32") + monkeypatch.setattr( + "je_auto_control.utils.accessibility.backends.windows_backend" + "._is_available", lambda: True) + assert get_backend().name == "windows-uia" + + +def test_windows_without_comtypes_gets_a_refusal_naming_it(on_platform, + monkeypatch): + on_platform("win32") + monkeypatch.setattr( + "je_auto_control.utils.accessibility.backends.windows_backend" + "._is_available", lambda: False) + backend = get_backend() + assert isinstance(backend, NullAccessibilityBackend) + with pytest.raises(AccessibilityNotAvailableError, match="comtypes"): + backend.list_elements() + + +def test_a_mac_with_pyobjc_selects_the_ax_backend(on_platform, monkeypatch): + on_platform("darwin") + objc_stub.install(monkeypatch, objc_stub.World()) + _stub_module(monkeypatch, "ApplicationServices") + assert get_backend().name == "macos-ax" + + +def test_a_mac_without_pyobjc_gets_a_refusal_naming_it(on_platform, + monkeypatch): + on_platform("darwin") + monkeypatch.setitem(sys.modules, "ApplicationServices", None) + backend = get_backend() + with pytest.raises(AccessibilityNotAvailableError, match="pyobjc"): + backend.list_elements() + + +def test_a_linux_session_with_a_bus_selects_the_atspi_backend(on_platform, + monkeypatch): + on_platform("linux") + monkeypatch.setattr( + "je_auto_control.utils.accessibility.backends.linux_backend" + "._is_available", lambda: True) + assert get_backend().name == "linux-atspi" + + +def test_a_linux_session_with_no_bus_names_both_ways_it_can_be_missing( + on_platform, monkeypatch): + # AT-SPI is a bus, not a library, so "not installed" and "installed but + # nothing is bridged to it" look identical from here. + on_platform("linux") + monkeypatch.setattr( + "je_auto_control.utils.accessibility.backends.linux_backend" + "._is_available", lambda: False) + backend = get_backend() + with pytest.raises(AccessibilityNotAvailableError) as caught: + backend.list_elements() + message = str(caught.value) + assert "at-spi2-core" in message + assert "atk-bridge" in message + + +def test_an_unknown_platform_gets_a_refusal_naming_it(on_platform): + on_platform("sunos5") + with pytest.raises(AccessibilityNotAvailableError, match="sunos5"): + get_backend().list_elements() + + +def test_the_choice_is_made_once_and_cached(on_platform): + on_platform("sunos5") + assert get_backend() is get_backend() + + +def test_resetting_the_cache_re_detects(on_platform): + on_platform("sunos5") + first = get_backend() + reset_backend_cache() + assert get_backend() is not first + + +# --- what the base class refuses ---------------------------------------------- + +_REFUSALS = [ + ("get_value", lambda b: b.get_value()), + ("set_value", lambda b: b.set_value("x")), + ("invoke", lambda b: b.invoke()), + ("toggle", lambda b: b.toggle()), + ("read_table", lambda b: b.read_table()), + ("expand", lambda b: b.expand()), + ("collapse", lambda b: b.collapse()), + ("expand_state", lambda b: b.expand_state()), + ("select_item", lambda b: b.select_item()), + ("get_range", lambda b: b.get_range()), + ("set_range_value", lambda b: b.set_range_value(1.0)), + ("scroll_into_view", lambda b: b.scroll_into_view()), + ("document_text", lambda b: b.document_text()), + ("selected_text", lambda b: b.selected_text()), + ("visible_text", lambda b: b.visible_text()), + ("find_text", lambda b: b.find_text("x")), + ("select_text", lambda b: b.select_text("x")), + ("text_attributes", lambda b: b.text_attributes()), + ("set_focus", lambda b: b.set_focus()), + ("find_virtual_item", lambda b: b.find_virtual_item("x")), + ("get_properties", lambda b: b.get_properties()), + ("get_state", lambda b: b.get_state()), + ("get_table_headers", lambda b: b.get_table_headers()), + ("get_grid_cell", lambda b: b.get_grid_cell(0, 0)), + ("move_element", lambda b: b.move_element(1.0, 2.0)), + ("resize_element", lambda b: b.resize_element(1.0, 2.0)), + ("set_window_state", lambda b: b.set_window_state("normal")), + ("window_interaction_state", lambda b: b.window_interaction_state()), + ("legacy_info", lambda b: b.legacy_info()), + ("legacy_default_action", lambda b: b.legacy_default_action()), + ("get_selection", lambda b: b.get_selection()), + ("list_views", lambda b: b.list_views()), + ("set_view", lambda b: b.set_view("Details")), + ("wait_for_focus_change", lambda b: b.wait_for_focus_change(0.1)), +] + + +@pytest.mark.parametrize("operation,call", + _REFUSALS, ids=[name for name, _ in _REFUSALS]) +def test_an_unimplemented_pattern_says_which_one_and_whose(operation, call): + # "This backend cannot do it" and "the control is not there" are + # different answers, and a caller that cannot tell them apart retries + # forever. The message carries both halves. + backend = AccessibilityBackend() + with pytest.raises(AccessibilityNotAvailableError) as caught: + call(backend) + assert operation in str(caught.value) + assert backend.name in str(caught.value) + + +def test_the_base_class_has_no_listing_of_its_own(): + with pytest.raises(NotImplementedError): + AccessibilityBackend().list_elements() + + +def test_the_null_backend_refuses_to_list_with_its_reason(): + with pytest.raises(AccessibilityNotAvailableError, match="no display"): + NullAccessibilityBackend("no display").list_elements() + + +def test_a_null_backend_with_no_reason_still_says_something(): + with pytest.raises(AccessibilityNotAvailableError) as caught: + NullAccessibilityBackend().list_elements() + assert str(caught.value) diff --git a/test/unit_test/headless/test_accessibility_linux.py b/test/unit_test/headless/test_accessibility_linux.py index c8340500..ea63a9e9 100644 --- a/test/unit_test/headless/test_accessibility_linux.py +++ b/test/unit_test/headless/test_accessibility_linux.py @@ -33,6 +33,13 @@ def __init__(self, tree=None, names=None, roles=None, extents=None, self.roles = roles or {} self.extents_map = extents or {} self.states = states or {} + self.texts = {} + self.numbers = {} + self.errors = {} # reference -> DBusError to raise on a write + self.actions = [] + self.written = [] + self.focused = [] + self.write_result = True self.entered = 0 self.exited = 0 @@ -65,10 +72,32 @@ def extents(self, reference): return self.extents_map.get(reference, (0, 0, 0, 0)) def text(self, reference): - return None + if reference in self.errors: + raise self.errors[reference] + return self.texts.get(reference) def number(self, reference): - return None + return self.numbers.get(reference) + + # --- writes, recorded rather than sent --------------------------------- + + def do_action(self, reference, index=0): + if reference in self.errors: + raise self.errors[reference] + self.actions.append((reference, index)) + return self.write_result + + def set_text(self, reference, value): + if reference in self.errors: + raise self.errors[reference] + self.written.append((reference, value)) + return self.write_result + + def grab_focus(self, reference): + if reference in self.errors: + raise self.errors[reference] + self.focused.append(reference) + return self.write_result APP = ("app", "/app") @@ -231,3 +260,247 @@ def call(self, *_args, **_kwargs): connection = atspi._AtspiConnection() connection._bus = TwoWordBus() assert connection.state(BUTTON) == (1 << 8) | (1 << 32) + + +# --- control patterns ------------------------------------------------------ +# +# The five object-level actions all share one shape: find the reference the +# caller described, then make exactly one AT-SPI call on it. What is worth +# pinning is what happens when the find comes back empty -- every one of them +# has to answer "no", because the caller cannot tell a control that refused +# from a control that was never there, and will otherwise retry forever. + + +def test_get_value_prefers_the_text_interface(backend): + backend.connection.texts[BUTTON] = "typed" + backend.connection.numbers[BUTTON] = 0.5 + assert backend.get_value(name="OK") == "typed" + + +def test_get_value_falls_back_to_a_numeric_value(backend): + # A slider has no text; its value is a double on the Value interface. + backend.connection.numbers[BUTTON] = 0.75 + assert backend.get_value(name="OK") == "0.75" + + +def test_get_value_of_a_control_with_neither_is_none(backend): + assert backend.get_value(name="OK") is None + + +def test_get_value_of_a_control_that_is_not_there_is_none(backend): + assert backend.get_value(name="Cancel") is None + + +def test_get_value_can_be_scoped_to_one_application(backend): + backend.connection.texts[BUTTON] = "typed" + assert backend.get_value(name="OK", app_name="zenity") == "typed" + assert backend.get_value(name="OK", app_name="gedit") is None + + +def test_get_value_matches_a_substring_when_asked(backend): + # Real interfaces label controls "Save(&S)" and "OK "; exact stays + # the default, so the same lower-case needle finds nothing without + # `contains`. + backend.connection.texts[BUTTON] = "typed" + assert backend.get_value(name="ok", contains=True) == "typed" + assert backend.get_value(name="ok") is None + + +def test_set_value_writes_through_the_editable_interface(backend): + assert backend.set_value("hello", name="OK") is True + assert backend.connection.written == [(BUTTON, "hello")] + + +def test_set_value_on_a_control_that_is_not_there_reports_failure(backend): + assert backend.set_value("hello", name="Cancel") is False + assert backend.connection.written == [] + + +def test_set_value_the_control_refuses_reports_failure(backend): + backend.connection.write_result = False + assert backend.set_value("hello", name="OK") is False + + +def test_a_write_that_fails_on_the_bus_reports_failure(backend): + backend.connection.errors[BUTTON] = DBusError("no EditableText") + assert backend.set_value("hello", name="OK") is False + + +def test_invoke_performs_the_first_action(backend): + assert backend.invoke(name="OK") is True + assert backend.connection.actions == [(BUTTON, 0)] + + +def test_invoke_on_a_control_that_is_not_there_reports_failure(backend): + assert backend.invoke(name="Cancel") is False + + +def test_an_invoke_that_fails_on_the_bus_reports_failure(backend): + backend.connection.errors[BUTTON] = DBusError("no Action") + assert backend.invoke(name="OK") is False + + +def test_set_focus_grabs_it_through_the_component_interface(backend): + assert backend.set_focus(name="OK") is True + assert backend.connection.focused == [BUTTON] + + +def test_set_focus_on_a_control_that_is_not_there_reports_failure(backend): + assert backend.set_focus(name="Cancel") is False + + +def test_a_focus_grab_that_fails_on_the_bus_reports_failure(backend): + backend.connection.errors[BUTTON] = DBusError("no Component") + assert backend.set_focus(name="OK") is False + + +def test_get_state_reports_the_three_bits_it_reads(backend): + backend.connection.states[BUTTON] = (1 << 8) | (1 << 12) | (1 << 25) + state = backend.get_state(name="OK") + assert state == {"enabled": True, "focused": True, "selected": True} + + +def test_get_state_reports_false_for_bits_that_are_clear(backend): + backend.connection.states[BUTTON] = 0 + assert backend.get_state(name="OK") == { + "enabled": False, "focused": False, "selected": False, + } + + +def test_get_state_carries_a_value_only_when_the_control_has_one(backend): + # An absent key and an empty value are different answers: the first says + # the control has no such concept, the second that it is empty. + assert "value" not in backend.get_state(name="OK") + backend.connection.texts[BUTTON] = "" + assert backend.get_state(name="OK")["value"] == "" + + +def test_get_state_carries_a_number_only_when_the_control_has_one(backend): + assert "number" not in backend.get_state(name="OK") + backend.connection.numbers[BUTTON] = 0.0 + assert backend.get_state(name="OK")["number"] == 0.0 + + +def test_get_state_of_a_control_that_is_not_there_is_none(backend): + assert backend.get_state(name="Cancel") is None + + +def test_a_control_pattern_needs_a_backend_that_is_available(): + instance = atspi.LinuxAccessibilityBackend.__new__( + atspi.LinuxAccessibilityBackend) + instance.available = False + for call in (lambda: instance.get_value(name="OK"), + lambda: instance.set_value("x", name="OK"), + lambda: instance.invoke(name="OK"), + lambda: instance.set_focus(name="OK"), + lambda: instance.get_state(name="OK")): + with pytest.raises(AccessibilityNotAvailableError): + call() + + +def test_the_search_gives_up_at_a_bounded_depth(monkeypatch): + """A tree that never ends must not recurse until Python gives up.""" + deep = ("app", "/deep") + connection = FakeConnection( + tree={("registry", "/root"): [APP], APP: [deep], deep: [deep]}, + names={APP: "zenity"}, + ) + monkeypatch.setattr(atspi, "_AtspiConnection", lambda: connection) + instance = atspi.LinuxAccessibilityBackend.__new__( + atspi.LinuxAccessibilityBackend) + instance.available = True + assert instance.get_value(name="nothing here") is None + + +def test_a_branch_the_bus_refuses_ends_the_search_there(monkeypatch): + class RefusingConnection(FakeConnection): + def children(self, reference): + if reference == APP: + raise DBusError("BadWindow") + return super().children(reference) + + connection = RefusingConnection( + tree={("registry", "/root"): [APP]}, names={APP: "zenity"}) + monkeypatch.setattr(atspi, "_AtspiConnection", lambda: connection) + instance = atspi.LinuxAccessibilityBackend.__new__( + atspi.LinuxAccessibilityBackend) + instance.available = True + assert instance.get_value(name="OK") is None + + +def test_an_application_whose_name_cannot_be_read_is_still_walked(monkeypatch): + class NamelessConnection(FakeConnection): + def property(self, reference, name, interface=None): + if reference == APP: + raise DBusError("gone") + return super().property(reference, name, interface) + + connection = NamelessConnection( + tree={("registry", "/root"): [APP], APP: [BUTTON]}, + names={BUTTON: "OK"}, roles={BUTTON: "push button"}) + monkeypatch.setattr(atspi, "_AtspiConnection", lambda: connection) + instance = atspi.LinuxAccessibilityBackend.__new__( + atspi.LinuxAccessibilityBackend) + instance.available = True + [element] = instance.list_elements() + assert element.app_name == "" + + +def test_closing_a_connection_twice_is_harmless(): + """`__exit__` runs on the way out of a `with` and again on a retry.""" + connection = atspi._AtspiConnection() + connection.__exit__() + connection.__exit__() + assert connection._bus is None + + +def test_the_walk_stops_asking_further_applications_once_it_is_full( + monkeypatch): + second = ("other", "/app") + connection = FakeConnection( + tree={("registry", "/root"): [APP, second], + APP: [WINDOW], second: [BUTTON]}, + names={APP: "zenity", WINDOW: "dialog", second: "gedit", + BUTTON: "OK"}, + roles={WINDOW: "dialog", BUTTON: "push button"}, + ) + monkeypatch.setattr(atspi, "_AtspiConnection", lambda: connection) + instance = atspi.LinuxAccessibilityBackend.__new__( + atspi.LinuxAccessibilityBackend) + instance.available = True + assert len(instance.list_elements(max_results=1)) == 1 + + +def test_a_branch_the_bus_refuses_ends_that_branch_of_the_walk(monkeypatch): + class RefusingConnection(FakeConnection): + def children(self, reference): + if reference == WINDOW: + raise DBusError("the dialog closed") + return super().children(reference) + + connection = RefusingConnection( + tree={("registry", "/root"): [APP], APP: [WINDOW], WINDOW: [BUTTON]}, + names={APP: "zenity", WINDOW: "dialog", BUTTON: "OK"}, + roles={WINDOW: "dialog", BUTTON: "push button"}, + ) + monkeypatch.setattr(atspi, "_AtspiConnection", lambda: connection) + instance = atspi.LinuxAccessibilityBackend.__new__( + atspi.LinuxAccessibilityBackend) + instance.available = True + # The window itself was listed before its children were asked for. + assert [e.name for e in instance.list_elements()] == ["dialog"] + + +def test_the_walk_stops_mid_application_once_it_is_full(monkeypatch): + siblings = [("app", f"/b{index}") for index in range(4)] + connection = FakeConnection( + tree={("registry", "/root"): [APP], APP: siblings}, + names={APP: "zenity", **{ref: f"b{index}" + for index, ref in enumerate(siblings)}}, + roles={ref: "push button" for ref in siblings}, + ) + monkeypatch.setattr(atspi, "_AtspiConnection", lambda: connection) + instance = atspi.LinuxAccessibilityBackend.__new__( + atspi.LinuxAccessibilityBackend) + instance.available = True + assert len(instance.list_elements(max_results=2)) == 2 diff --git a/test/unit_test/headless/test_accessibility_linux_bus.py b/test/unit_test/headless/test_accessibility_linux_bus.py new file mode 100644 index 00000000..560ea750 --- /dev/null +++ b/test/unit_test/headless/test_accessibility_linux_bus.py @@ -0,0 +1,390 @@ +"""The AT-SPI wire itself: what the backend actually sends down the bus. + +`test_accessibility_linux.py` replaces `_AtspiConnection` with a fake tree, +which is the right shape for testing the *walk* -- and it leaves the whole +D-Bus call layer beneath it unexecuted. That is where the protocol lives, and +the protocol is where the surprises are: + +* **The accessibility bus is not the session bus.** Its address comes from + `org.a11y.Bus.GetAddress` on the session bus, and everything after that + happens on a second connection to the address that call returns. Talking + AT-SPI to the session bus reaches nobody. +* **An accessible is a *pair*.** The bus name of the owning application plus + an object path inside it -- so references are `(sender, path)` tuples all + the way down, and a call addressed with only a path goes to the wrong + process. +* **The state bitfield arrives as two 32-bit words.** AT-SPI does not send + one 64-bit value, so reading only the first word silently drops every + state above bit 31 -- including SELECTED, which this backend reports. +* **Not implementing an interface is not an error.** A plain text node has no + Component and therefore no rectangle; a walk that treated that as a failure + would fail more often than it succeeded. + +`SessionBus` is replaced rather than `_AtspiConnection`: the whole point is +to run the code that builds those calls. What it is replaced by is a +recorder, so each test can state the exact `(destination, path, interface, +member, signature, body)` that went out. + +The bus is also where this backend gets its blast radius: `_is_available()` +opens a connection at import-decision time on every Linux desktop, so its +failure paths matter as much as its success one. +""" +from __future__ import annotations + +import pytest + +from je_auto_control.utils.accessibility.backends import linux_backend as atspi +from je_auto_control.utils.accessibility.backends.linux_backend import ( + _AtspiConnection, LinuxAccessibilityBackend, +) +from je_auto_control.utils.dbus_client import DBusError, Variant + +ROOT = ("org.a11y.atspi.Registry", "/org/a11y/atspi/accessible/root") +APP = ("app.bus.name", "/org/a11y/atspi/accessible/1") +BUTTON = ("app.bus.name", "/org/a11y/atspi/accessible/2") + + +class _Bus: + """A session bus that answers scripted replies and records the calls.""" + + instances = [] + + def __init__(self, address=None) -> None: + self.address = address + self.calls = [] + self.connected = False + self.closed = False + self.replies = {} + self.errors = {} + self.connect_error = None + _Bus.instances.append(self) + + # -- the SessionBus surface -- + def connect(self): + if self.connect_error is not None: + raise self.connect_error + self.connected = True + + def close(self): + self.closed = True + + def __enter__(self): + self.connect() + return self + + def __exit__(self, *_exception): + self.close() + + def call(self, destination, path, interface, member, signature, body, + timeout=25.0): + self.calls.append((destination, path, interface, member, signature, + list(body), timeout)) + key = (path, interface, member) + if key in self.errors: + raise self.errors[key] + if (interface, member) in self.errors: + raise self.errors[(interface, member)] + if key in self.replies: + return self.replies[key] + return self.replies.get((interface, member), []) + + +@pytest.fixture +def bus(monkeypatch): + """One bus for both connections, so a test can script either.""" + _Bus.instances = [] + instance = _Bus() + instance.replies[("org.a11y.Bus", "GetAddress")] = ["unix:path=/a11y"] + monkeypatch.setattr(atspi, "SessionBus", lambda address=None: instance) + yield instance + _Bus.instances = [] + + +@pytest.fixture +def connection(bus): + with _AtspiConnection() as opened: + yield opened + + +def _call(bus, index=-1): + """One recorded call, without the timeout.""" + return bus.calls[index][:6] + + +# --- finding the accessibility bus -------------------------------------------- + +def test_the_address_is_asked_of_the_session_bus(bus): + with _AtspiConnection(): + pass + destination, path, interface, member, signature, body = _call(bus, 0) + assert (destination, path) == ("org.a11y.Bus", "/org/a11y/bus") + assert (interface, member) == ("org.a11y.Bus", "GetAddress") + assert (signature, body) == ("", []) + + +def test_the_connection_opens_on_the_address_it_was_given(monkeypatch): + # A second connection, to a second bus. Reusing the session bus would + # send AT-SPI calls somewhere that does not speak it. + opened = [] + + def _factory(address=None): + instance = _Bus(address) + instance.replies[("org.a11y.Bus", "GetAddress")] = ["unix:path=/a11y"] + opened.append(instance) + return instance + + monkeypatch.setattr(atspi, "SessionBus", _factory) + with _AtspiConnection(): + pass + assert [b.address for b in opened] == [None, "unix:path=/a11y"] + + +def test_the_connection_is_closed_on_the_way_out(bus): + with _AtspiConnection(): + pass + assert bus.closed is True + + +@pytest.mark.parametrize("reply", [[], [42], None]) +def test_an_address_that_is_not_a_string_is_an_error(bus, reply): + bus.replies[("org.a11y.Bus", "GetAddress")] = reply + with pytest.raises(DBusError, match="GetAddress"): + with _AtspiConnection(): + pass + + +def test_using_the_connection_unopened_says_which_mistake_it_was(bus): + # The class is a context manager on purpose; forgetting the `with` is a + # programming error worth naming rather than an AttributeError. + with pytest.raises(DBusError, match="context manager"): + _AtspiConnection().children(ROOT) + + +def test_the_root_is_the_registry(connection): + assert connection.root == ROOT + + +# --- reads -------------------------------------------------------------------- + +def test_children_are_read_as_sender_and_path_pairs(connection, bus): + bus.replies[("org.a11y.atspi.Accessible", "GetChildren")] = [ + [["app.bus.name", "/org/a11y/atspi/accessible/1"], + ["app.bus.name", "/org/a11y/atspi/accessible/2"]], + ] + assert connection.children(ROOT) == [APP, BUTTON] + + +def test_a_child_entry_that_is_not_a_pair_is_dropped(connection, bus): + # A malformed reply is a bug in the application, not a reason to fail the + # whole walk of everything else on the desktop. + bus.replies[("org.a11y.atspi.Accessible", "GetChildren")] = [ + ["not a pair", ["app.bus.name", "/1"], ["only-one"]], + ] + assert connection.children(ROOT) == [("app.bus.name", "/1")] + + +def test_an_empty_reply_reads_as_no_children(connection, bus): + assert connection.children(ROOT) == [] + + +def test_a_call_is_addressed_to_the_owning_application(connection, bus): + connection.children(BUTTON) + destination, path, interface, member, _signature, _body = _call(bus) + assert destination == "app.bus.name", "the sender half of the reference" + assert path == "/org/a11y/atspi/accessible/2" + assert (interface, member) == ("org.a11y.atspi.Accessible", "GetChildren") + + +def test_a_property_is_unwrapped_from_its_variant(connection, bus): + bus.replies[("org.freedesktop.DBus.Properties", "Get")] = [ + Variant("s", "OK"), + ] + assert connection.property(BUTTON, "Name") == "OK" + _d, _p, interface, member, signature, body = _call(bus) + assert (interface, member) == ("org.freedesktop.DBus.Properties", "Get") + assert (signature, body) == ("ss", ["org.a11y.atspi.Accessible", "Name"]) + + +def test_a_property_that_is_not_a_variant_is_passed_through(connection, bus): + bus.replies[("org.freedesktop.DBus.Properties", "Get")] = ["plain"] + assert connection.property(BUTTON, "Name") == "plain" + + +def test_a_property_with_no_reply_reads_as_none(connection, bus): + assert connection.property(BUTTON, "Name") is None + + +def test_a_property_can_be_asked_of_another_interface(connection, bus): + bus.replies[("org.freedesktop.DBus.Properties", "Get")] = [Variant("d", 5)] + connection.property(BUTTON, "CurrentValue", "org.a11y.atspi.Value") + _d, _p, _i, _m, _signature, body = _call(bus) + assert body == ["org.a11y.atspi.Value", "CurrentValue"] + + +def test_the_role_name_is_read_as_text(connection, bus): + bus.replies[("org.a11y.atspi.Accessible", "GetRoleName")] = ["push button"] + assert connection.role_name(BUTTON) == "push button" + + +def test_a_missing_role_name_reads_as_empty(connection, bus): + assert connection.role_name(BUTTON) == "" + + +def test_the_state_is_assembled_from_both_words(connection, bus): + # AT-SPI sends the bitfield as two 32-bit words. Reading only the first + # would drop every state above bit 31 -- SELECTED among them. + bus.replies[("org.a11y.atspi.Accessible", "GetState")] = [[0, 1 << 25]] + assert connection.state(BUTTON) == 1 << (32 + 25) + + +def test_the_low_word_lands_where_the_backend_looks_for_enabled(connection, + bus): + bus.replies[("org.a11y.atspi.Accessible", "GetState")] = [[1 << 8, 0]] + assert connection.state(BUTTON) & (1 << 8) + + +def test_extra_words_beyond_the_first_two_are_ignored(connection, bus): + bus.replies[("org.a11y.atspi.Accessible", "GetState")] = [[0, 0, 0xFF]] + assert connection.state(BUTTON) == 0 + + +@pytest.mark.parametrize("reply", [[], [[]], [None]]) +def test_a_state_reply_with_nothing_in_it_reads_as_zero(connection, bus, + reply): + bus.replies[("org.a11y.atspi.Accessible", "GetState")] = reply + assert connection.state(BUTTON) == 0 + + +def test_extents_are_asked_in_screen_coordinates(connection, bus): + # Screen is the only coordinate space whose numbers mean anything to a + # caller that is about to click them. + bus.replies[("org.a11y.atspi.Component", "GetExtents")] = [[1, 2, 3, 4]] + assert connection.extents(BUTTON) == (1, 2, 3, 4) + _d, _p, _i, _m, signature, body = _call(bus) + assert (signature, body) == ("u", [0]) + + +def test_an_accessible_with_no_component_interface_has_no_rectangle( + connection, bus): + # A plain text node implements no Component; that is not an error. + bus.errors[("org.a11y.atspi.Component", "GetExtents")] = DBusError("no") + assert connection.extents(BUTTON) == (0, 0, 0, 0) + + +@pytest.mark.parametrize("reply", [[], [[1, 2]]]) +def test_a_short_extents_reply_reads_as_the_origin(connection, bus, reply): + bus.replies[("org.a11y.atspi.Component", "GetExtents")] = reply + assert connection.extents(BUTTON) == (0, 0, 0, 0) + + +def test_text_is_read_over_the_whole_range(connection, bus): + bus.replies[("org.a11y.atspi.Text", "GetText")] = ["hello"] + assert connection.text(BUTTON) == "hello" + _d, _p, _i, _m, signature, body = _call(bus) + assert (signature, body) == ("ii", [0, -1]), "0 to -1 is 'all of it'" + + +def test_an_accessible_with_no_text_interface_has_no_text(connection, bus): + bus.errors[("org.a11y.atspi.Text", "GetText")] = DBusError("no Text") + assert connection.text(BUTTON) is None + + +def test_an_empty_text_reply_reads_as_none(connection, bus): + assert connection.text(BUTTON) is None + + +def test_a_numeric_value_is_read_from_the_value_interface(connection, bus): + bus.replies[("org.freedesktop.DBus.Properties", "Get")] = [ + Variant("d", 0.75), + ] + assert connection.number(BUTTON) == 0.75 + + +@pytest.mark.parametrize("reply,error", [ + ([Variant("s", "not a number")], None), + ([None], None), + (None, DBusError("no Value interface")), +]) +def test_a_control_with_no_number_reads_as_none(connection, bus, reply, error): + if error is not None: + bus.errors[("org.freedesktop.DBus.Properties", "Get")] = error + else: + bus.replies[("org.freedesktop.DBus.Properties", "Get")] = reply + assert connection.number(BUTTON) is None + + +# --- writes ------------------------------------------------------------------- + +def test_an_action_is_performed_by_index(connection, bus): + bus.replies[("org.a11y.atspi.Action", "DoAction")] = [True] + assert connection.do_action(BUTTON, 0) is True + _d, _p, interface, member, signature, body = _call(bus) + assert (interface, member) == ("org.a11y.atspi.Action", "DoAction") + assert (signature, body) == ("i", [0]) + + +def test_an_action_the_control_refuses_reports_failure(connection, bus): + bus.replies[("org.a11y.atspi.Action", "DoAction")] = [False] + assert connection.do_action(BUTTON) is False + + +def test_an_action_with_no_reply_reports_failure(connection, bus): + assert connection.do_action(BUTTON) is False + + +def test_text_is_written_through_the_editable_interface(connection, bus): + bus.replies[("org.a11y.atspi.EditableText", "SetTextContents")] = [True] + assert connection.set_text(BUTTON, "typed") is True + _d, _p, interface, member, signature, body = _call(bus) + assert (interface, member) == ("org.a11y.atspi.EditableText", + "SetTextContents") + assert (signature, body) == ("s", ["typed"]) + + +def test_a_write_the_control_refuses_reports_failure(connection, bus): + bus.replies[("org.a11y.atspi.EditableText", "SetTextContents")] = [False] + assert connection.set_text(BUTTON, "typed") is False + + +def test_focus_is_grabbed_through_the_component_interface(connection, bus): + bus.replies[("org.a11y.atspi.Component", "GrabFocus")] = [True] + assert connection.grab_focus(BUTTON) is True + _d, _p, interface, member, _s, _b = _call(bus) + assert (interface, member) == ("org.a11y.atspi.Component", "GrabFocus") + + +def test_a_focus_grab_the_control_refuses_reports_failure(connection, bus): + assert connection.grab_focus(BUTTON) is False + + +# --- availability ------------------------------------------------------------- + +def test_availability_is_decided_by_reaching_the_root(monkeypatch, bus): + monkeypatch.setattr(atspi.os, "name", "posix") + bus.replies[("org.a11y.atspi.Accessible", "GetChildren")] = [[]] + assert atspi._is_available() is True + + +def test_a_desktop_with_no_accessibility_bus_is_unavailable(monkeypatch, bus): + monkeypatch.setattr(atspi.os, "name", "posix") + bus.connect_error = DBusError("connection refused") + assert atspi._is_available() is False + + +def test_a_bus_that_cannot_be_reached_at_all_is_unavailable(monkeypatch, bus): + monkeypatch.setattr(atspi.os, "name", "posix") + bus.connect_error = OSError("no such file or directory") + assert atspi._is_available() is False + + +def test_off_posix_no_bus_is_even_attempted(monkeypatch, bus): + monkeypatch.setattr(atspi.os, "name", "nt") + assert atspi._is_available() is False + assert bus.calls == [], "not one D-Bus round trip on Windows" + + +def test_the_backend_takes_its_availability_from_the_probe(monkeypatch): + monkeypatch.setattr(atspi, "_is_available", lambda: True) + assert LinuxAccessibilityBackend().available is True + monkeypatch.setattr(atspi, "_is_available", lambda: False) + assert LinuxAccessibilityBackend().available is False diff --git a/test/unit_test/headless/test_accessibility_macos.py b/test/unit_test/headless/test_accessibility_macos.py new file mode 100644 index 00000000..871d10c2 --- /dev/null +++ b/test/unit_test/headless/test_accessibility_macos.py @@ -0,0 +1,314 @@ +"""Walking the macOS accessibility tree, from any square rather than Darwin. + +`backends/macos_backend.py` read 0% everywhere. Its pyobjc imports are inside +the methods, so the module loads on any platform and stubs in `sys.modules` +(`_pyobjc_stub.py`) drive the whole walk from all nine CI squares. + +What the walk has to get right: + +* **The default scope is the *active* application.** Enumerating every + running application's whole tree is thousands of nodes for a caller who + asked about the window in front of them, so an inactive application is + skipped -- unless it was named, which is the one case where the caller + clearly meant it. +* **`max_results` bounds the recursion, not just the answer.** The tree is + deep and the AX API is a round trip per node; a walk that collected + everything and sliced afterwards would pay for the whole desktop. +* **One application's AX failure must not end the enumeration.** Accessibility + is granted per process and revoked at any time, and an application that + refuses is a normal event, not a reason to return nothing. +* **A node with neither role nor title is not an element.** AX reports + structural nodes with nothing on them; carrying them would fill the answer + with rows a caller cannot match on. +""" +from __future__ import annotations + +import types + +import pytest + +from headless import _pyobjc_stub as objc_stub +from headless._pyobjc_stub import AX_FAILURE, AXElement +from je_auto_control.utils.accessibility.backends import macos_backend as mac +from je_auto_control.utils.accessibility.backends.macos_backend import ( + MacOSAccessibilityBackend, _extract_bounds, +) +from je_auto_control.utils.accessibility.element import ( + AccessibilityNotAvailableError, +) + + +class _App: + """One entry of `NSWorkspace.runningApplications()`.""" + + def __init__(self, name: str, pid: int, active: bool = True) -> None: + self._name = name + self._pid = pid + self._active = active + + def isActive(self): # noqa: N802 # reason: the AppKit name + return self._active + + def localizedName(self): # noqa: N802 # reason: the AppKit name + return self._name + + def processIdentifier(self): # noqa: N802 # reason: the AppKit name + return self._pid + + +def _element(role=None, title=None, position=None, size=None, children=None): + attributes = {} + if role is not None: + attributes["AXRole"] = role + if title is not None: + attributes["AXTitle"] = title + if position is not None: + attributes["AXPosition"] = position + if size is not None: + attributes["AXSize"] = size + if children is not None: + attributes["AXChildren"] = children + return AXElement(**attributes) + + +class _World(objc_stub.World): + """The AT-SPI-free half of the pyobjc stub: apps, and a tree per pid.""" + + def __init__(self, apps=None, trees=None) -> None: + super().__init__() + self.apps = list(apps or []) + self.trees = dict(trees or {}) # pid -> root AXElement + self.walk_errors = {} # pid -> exception + + def ax_application(self, pid): + if pid in self.walk_errors: + raise self.walk_errors[pid] + return self.trees.get(int(pid), AXElement()) + + def ax_copy_attribute(self, element, attribute, placeholder): + # An AX read can raise rather than answer with an error code: the + # pyobjc bridge turns some failures into exceptions. + error = getattr(element, "raises", None) + if error is not None: + raise error + return super().ax_copy_attribute(element, attribute, placeholder) + + +@pytest.fixture +def install(monkeypatch): + def _install(world: _World) -> MacOSAccessibilityBackend: + objc_stub.install(monkeypatch, world) + appkit = types.ModuleType("AppKit") + appkit.NSWorkspace = types.SimpleNamespace( + sharedWorkspace=lambda: types.SimpleNamespace( + runningApplications=lambda: list(world.apps))) + monkeypatch.setitem(__import__("sys").modules, "AppKit", appkit) + backend = MacOSAccessibilityBackend() + assert backend.available + return backend + return _install + + +# --- availability ------------------------------------------------------------- + +def test_a_mac_without_pyobjc_refuses_and_names_it(monkeypatch): + monkeypatch.setattr(mac, "_is_available", lambda: False) + backend = MacOSAccessibilityBackend() + with pytest.raises(AccessibilityNotAvailableError, match="pyobjc"): + backend.list_elements() + + +def test_the_probe_reports_what_it_could_import(monkeypatch): + import sys + monkeypatch.setitem(sys.modules, "ApplicationServices", + types.ModuleType("ApplicationServices")) + monkeypatch.setitem(sys.modules, "AppKit", types.ModuleType("AppKit")) + assert mac._is_available() is True + monkeypatch.setitem(sys.modules, "ApplicationServices", None) + assert mac._is_available() is False + + +def test_the_backend_names_the_api_it_walks(install): + assert install(_World()).name == "macos-ax" + + +# --- scoping ------------------------------------------------------------------ + +def test_only_the_active_application_is_walked_by_default(install): + world = _World( + apps=[_App("Safari", 1, active=True), _App("Mail", 2, active=False)], + trees={1: _element(role="AXWindow", title="Safari window"), + 2: _element(role="AXWindow", title="Mail window")}, + ) + names = [e.name for e in install(world).list_elements()] + assert names == ["Safari window"] + + +def test_naming_an_application_reaches_it_even_when_it_is_not_active(install): + world = _World( + apps=[_App("Safari", 1, active=True), _App("Mail", 2, active=False)], + trees={1: _element(role="AXWindow", title="Safari window"), + 2: _element(role="AXWindow", title="Mail window")}, + ) + elements = install(world).list_elements(app_name="Mail") + assert [e.name for e in elements] == ["Mail window"] + assert elements[0].app_name == "Mail" + + +def test_naming_an_application_that_is_not_running_finds_nothing(install): + world = _World(apps=[_App("Safari", 1)], + trees={1: _element(role="AXWindow", title="w")}) + assert install(world).list_elements(app_name="Mail") == [] + + +def test_an_application_with_no_localized_name_is_still_walked(install): + world = _World(apps=[_App(None, 1)], + trees={1: _element(role="AXWindow", title="untitled")}) + [element] = install(world).list_elements() + assert element.app_name == "" + + +def test_the_owning_process_is_carried_on_every_element(install): + world = _World(apps=[_App("Safari", 501)], + trees={501: _element(role="AXWindow", title="w")}) + [element] = install(world).list_elements() + assert element.process_id == 501 + + +# --- the walk ----------------------------------------------------------------- + +def test_the_walk_is_depth_first_through_the_children(install): + leaf = _element(role="AXButton", title="OK") + group = _element(role="AXGroup", title="Buttons", children=[leaf]) + root = _element(role="AXWindow", title="Dialog", children=[group]) + world = _World(apps=[_App("Safari", 1)], trees={1: root}) + names = [e.name for e in install(world).list_elements()] + assert names == ["Dialog", "Buttons", "OK"] + + +def test_a_node_with_neither_role_nor_title_is_not_an_element(install): + structural = _element(children=[_element(role="AXButton", title="OK")]) + world = _World(apps=[_App("Safari", 1)], trees={1: structural}) + names = [e.name for e in install(world).list_elements()] + assert names == ["OK"], "the structural node is walked through, not listed" + + +def test_a_node_with_only_a_role_is_still_an_element(install): + world = _World(apps=[_App("Safari", 1)], + trees={1: _element(role="AXWindow")}) + [element] = install(world).list_elements() + assert (element.role, element.name) == ("AXWindow", "") + + +def test_the_walk_stops_at_max_results(install): + children = [_element(role="AXButton", title=f"b{index}") + for index in range(10)] + root = _element(role="AXWindow", title="Dialog", children=children) + world = _World(apps=[_App("Safari", 1)], trees={1: root}) + assert len(install(world).list_elements(max_results=3)) == 3 + + +def test_max_results_bounds_the_recursion_not_just_the_answer(install): + # Every node is a round trip; collecting the desktop and slicing after + # would pay for all of it. + deep = _element(role="AXButton", title="deep") + nested = _element(role="AXGroup", title="g", children=[deep]) + root = _element(role="AXWindow", title="Dialog", children=[nested]) + world = _World(apps=[_App("Safari", 1)], trees={1: root}) + install(world).list_elements(max_results=1) + assert "AXChildren" not in nested.attributes or not nested.actions + + +def test_the_walk_stops_asking_further_applications_once_it_is_full(install): + world = _World( + apps=[_App("Safari", 1), _App("Mail", 2)], + trees={1: _element(role="AXWindow", title="one"), + 2: _element(role="AXWindow", title="two")}, + ) + assert len(install(world).list_elements(max_results=1)) == 1 + + +def test_a_node_that_refuses_to_list_children_ends_that_branch(install): + root = _element(role="AXWindow", title="Dialog") + root.read_error = AX_FAILURE + world = _World(apps=[_App("Safari", 1)], trees={1: root}) + # The node itself cannot be described either, so nothing comes back -- + # and, crucially, no exception does. + assert install(world).list_elements() == [] + + +def test_a_node_whose_read_raises_is_skipped_rather_than_fatal(install): + # pyobjc turns some AX failures into exceptions rather than error codes, + # and a control that vanished mid-walk is an ordinary event. + root = _element(role="AXWindow", title="Dialog") + root.raises = RuntimeError("AXError -25202") + world = _World(apps=[_App("Safari", 1)], trees={1: root}) + assert install(world).list_elements() == [] + + +def test_the_recursion_guard_holds_even_when_entered_full(install): + # The loop checks before it recurses, so this guard is only reachable by + # entering the walk with a full list -- which is what makes the helper + # safe to call from anywhere, including a future second caller. + world = _World(apps=[_App("Safari", 1)], + trees={1: _element(role="AXWindow", title="w")}) + backend = install(world) + import ApplicationServices as ax_module + results = ["already full"] + backend._walk(ax_module, _element(role="AXWindow", title="w"), + "Safari", 1, results, max_results=1) + assert results == ["already full"] + + +def test_one_application_failing_does_not_end_the_enumeration(install): + world = _World( + apps=[_App("Broken", 1), _App("Safari", 2)], + trees={2: _element(role="AXWindow", title="works")}, + ) + world.walk_errors[1] = RuntimeError("AXError -25211") + names = [e.name for e in install(world).list_elements(app_name=None)] + assert names == ["works"] + + +def test_a_window_title_scope_is_accepted_and_ignored(install): + # AX walks per application already; returning nothing because the scope + # cannot be honoured would be worse than returning the unscoped answer. + world = _World(apps=[_App("Safari", 1)], + trees={1: _element(role="AXWindow", title="Dialog")}) + elements = install(world).list_elements(window_title="something else") + assert [e.name for e in elements] == ["Dialog"] + + +# --- geometry ----------------------------------------------------------------- + +def test_an_elements_bounds_come_from_its_position_and_size(install): + world = _World(apps=[_App("Safari", 1)], trees={ + 1: _element(role="AXWindow", title="w", position=(10, 20), + size=(300, 400)), + }) + [element] = install(world).list_elements() + assert element.bounds == (10, 20, 300, 400) + + +@pytest.mark.parametrize("position,size", [ + (None, (1, 2)), ((1, 2), None), (None, None), +]) +def test_an_element_with_no_geometry_reads_as_the_origin(position, size): + # A control AX will not place is still worth reporting: the caller can + # match on its name even if it cannot click it. + assert _extract_bounds(position, size) == (0, 0, 0, 0) + + +@pytest.mark.parametrize("position,size", [ + ("not a pair", (1, 2)), + ((1, 2), "not a pair"), + ((1,), (1, 2)), + (("x", "y"), (1, 2)), +]) +def test_geometry_that_cannot_be_unpacked_reads_as_the_origin(position, size): + assert _extract_bounds(position, size) == (0, 0, 0, 0) + + +def test_geometry_is_narrowed_to_whole_pixels(): + # AX reports floats; bounds are pixels a caller is about to click. + assert _extract_bounds((1.7, 2.2), (3.9, 4.1)) == (1, 2, 3, 4) diff --git a/test/unit_test/headless/test_accessibility_windows_content.py b/test/unit_test/headless/test_accessibility_windows_content.py new file mode 100644 index 00000000..ee444d43 --- /dev/null +++ b/test/unit_test/headless/test_accessibility_windows_content.py @@ -0,0 +1,731 @@ +"""Reading structured content out of a Windows control. + +Split from `test_accessibility_windows_patterns.py`, which covers acting on a +control; this half is the reads that come back as more than one value -- a +table, a selection, a set of views, the MSAA bridge's fields, a virtualized +row, and the text patterns. + +What they have in common is a second layer of indirection. A grid hands back +a cell which carries its *own* pattern; a selection hands back an element +array which has to be walked by index; a text pattern hands back ranges. Each +of those is another cross-process object that can fail on its own, and the +answer for every one of them is the same: report what could be read, and say +"no" in the type the caller was promised rather than raising out of the +middle of a listing. + +The virtualized-item path is the one worth reading twice. A row 500 places +down a list does not exist as an element until it is realized, so finding it +and realizing it are two calls and skipping the second hands the caller +something that is not there yet. +""" +from __future__ import annotations + +import sys +import types + +import pytest + +from headless._uia_doubles import ( + Automation, ElementArray, Pattern, RawElement, Rect, UiaModule, Unknown, + install_comtypes, +) +from je_auto_control.utils.accessibility.backends import ( + windows_backend as backend_module, +) +from je_auto_control.utils.accessibility.backends.windows_backend import ( + WindowsAccessibilityBackend, _header_names, _read_cell, _read_legacy, + _read_text_attributes, _view_name, +) + +VALUE = backend_module._UIA_VALUE_PATTERN_ID +INVOKE = backend_module._UIA_INVOKE_PATTERN_ID +TOGGLE = backend_module._UIA_TOGGLE_PATTERN_ID +GRID = backend_module._UIA_GRID_PATTERN_ID +GRID_ITEM = backend_module._UIA_GRIDITEM_PATTERN_ID +EXPAND = backend_module._UIA_EXPANDCOLLAPSE_PATTERN_ID +SELECTION_ITEM = backend_module._UIA_SELECTIONITEM_PATTERN_ID +RANGE = backend_module._UIA_RANGEVALUE_PATTERN_ID +SCROLL_ITEM = backend_module._UIA_SCROLLITEM_PATTERN_ID +TEXT = backend_module._UIA_TEXT_PATTERN_ID +ITEM_CONTAINER = backend_module._UIA_ITEMCONTAINER_PATTERN_ID +VIRTUALIZED = backend_module._UIA_VIRTUALIZEDITEM_PATTERN_ID +TABLE = backend_module._UIA_TABLE_PATTERN_ID +TRANSFORM = backend_module._UIA_TRANSFORM_PATTERN_ID +WINDOW = backend_module._UIA_WINDOW_PATTERN_ID +LEGACY = backend_module._UIA_LEGACYIACCESSIBLE_PATTERN_ID +SELECTION = backend_module._UIA_SELECTION_PATTERN_ID +MULTIPLE_VIEW = backend_module._UIA_MULTIPLEVIEW_PATTERN_ID + +_IS_PASSWORD = 30019 + + +@pytest.fixture(autouse=True) +def named_processes(monkeypatch): + monkeypatch.setattr(backend_module, "_process_name", + lambda pid: f"app{pid}.exe" if pid else "") + + +@pytest.fixture +def backend(monkeypatch): + monkeypatch.setattr(backend_module, "_is_available", lambda: True) + instance = WindowsAccessibilityBackend() + instance._automation = Automation() + instance._uia_module = UiaModule() + return instance + + +@pytest.fixture +def found(backend, monkeypatch): + """Answer every search with one raw element, and record what was asked.""" + state = {"raw": None, "filters": []} + + def _find_raw(name, role, app_name, automation_id, window_title=None, + contains=False): + state["filters"].append({"name": name, "role": role, + "app_name": app_name, + "automation_id": automation_id, + "window_title": window_title, + "contains": contains}) + return state["raw"] + + monkeypatch.setattr(backend, "_find_raw", _find_raw) + + def _set(raw): + state["raw"] = raw + return raw + + state["set"] = _set + return state + + +def _control(pattern_id=None, interface_name="", pattern=None, **kwargs): + """A raw element carrying at most one pattern.""" + patterns = {} + if pattern_id is not None: + patterns[pattern_id] = Unknown(interface_name, pattern or Pattern()) + kwargs.setdefault("rect", Rect(0, 0, 1, 1)) + return RawElement(patterns=patterns, **kwargs) + + +def _with(found, pattern_id, interface_name, pattern=None, **kwargs): + """Install a control carrying one pattern, and hand the pattern back.""" + pattern = pattern or Pattern() + found["set"](_control(pattern_id, interface_name, pattern, **kwargs)) + return pattern + + +# --- tables and grids --------------------------------------------------------- + +def _grid(rows, cols, cells=None, item_error=None): + lookup = dict(cells or {}) + + def _get_item(row, column): + if item_error is not None: + raise item_error + return lookup.get((row, column)) + + pattern = Pattern(CurrentRowCount=rows, CurrentColumnCount=cols) + pattern.GetItem = _get_item + return pattern + + +def test_a_table_is_read_row_by_row(backend, found): + cells = {(0, 0): types.SimpleNamespace(CurrentName="a"), + (0, 1): types.SimpleNamespace(CurrentName="b"), + (1, 0): types.SimpleNamespace(CurrentName="c"), + (1, 1): types.SimpleNamespace(CurrentName="d")} + _with(found, GRID, "IUIAutomationGridPattern", _grid(2, 2, cells)) + assert backend.read_table(name="Grid") == [["a", "b"], ["c", "d"]] + + +def test_a_cell_that_is_not_there_reads_as_empty(backend, found): + _with(found, GRID, "IUIAutomationGridPattern", _grid(1, 2, {})) + assert backend.read_table(name="Grid") == [["", ""]] + + +def test_a_cell_the_provider_refuses_reads_as_empty(backend, found): + _with(found, GRID, "IUIAutomationGridPattern", + _grid(1, 1, item_error=OSError("gone"))) + assert backend.read_table(name="Grid") == [[""]] + + +def test_a_table_whose_size_cannot_be_read_is_empty(backend, found): + _with(found, GRID, "IUIAutomationGridPattern", Pattern()) + assert backend.read_table(name="Grid") == [] + + +def test_a_control_that_is_not_a_grid_reads_as_no_table(backend, found): + found["set"](_control()) + assert backend.read_table(name="Grid") == [] + + +def test_a_control_that_is_not_there_reads_as_no_table(backend, found): + assert backend.read_table(name="Grid") == [] + + +def test_table_headers_are_read_from_both_axes(backend, found): + columns = ElementArray([types.SimpleNamespace(CurrentName="Name"), + types.SimpleNamespace(CurrentName="Size")]) + rows = ElementArray([types.SimpleNamespace(CurrentName="1")]) + pattern = Pattern() + pattern.GetCurrentColumnHeaders = lambda: columns + pattern.GetCurrentRowHeaders = lambda: rows + _with(found, TABLE, "IUIAutomationTablePattern", pattern) + assert backend.get_table_headers(name="Grid") == { + "columns": ["Name", "Size"], "rows": ["1"], + } + + +def test_headers_the_provider_refuses_read_as_none(backend, found): + pattern = Pattern() + pattern.errors["GetCurrentColumnHeaders"] = OSError("gone") + _with(found, TABLE, "IUIAutomationTablePattern", pattern) + assert backend.get_table_headers(name="Grid") is None + + +def test_a_control_with_no_table_pattern_has_no_headers(backend, found): + found["set"](_control()) + assert backend.get_table_headers(name="Grid") is None + + +def test_an_array_whose_length_cannot_be_read_is_empty(): + assert _header_names(ElementArray(length_error=OSError("gone"))) == [] + + +def test_an_array_entry_that_cannot_be_read_reads_as_empty(): + array = ElementArray([types.SimpleNamespace(CurrentName="a")], + element_error=OSError("gone")) + assert _header_names(array) == [""] + + +def test_a_grid_cell_carries_its_own_coordinates(backend, found): + cell = types.SimpleNamespace(CurrentName="value") + item = Unknown("IUIAutomationGridItemPattern", + Pattern(CurrentRow=3, CurrentColumn=4, CurrentRowSpan=2, + CurrentColumnSpan=1)) + cell_element = RawElement(patterns={GRID_ITEM: item}) + cell_element.CurrentName = "value" + del cell + _with(found, GRID, "IUIAutomationGridPattern", + _grid(1, 1, {(0, 0): cell_element})) + assert backend.get_grid_cell(0, 0, name="Grid") == { + "value": "value", "row": 3, "column": 4, + "row_span": 2, "column_span": 1, + } + + +def test_a_grid_cell_without_an_item_pattern_reports_where_it_was_asked_for(): + cell = types.SimpleNamespace(CurrentName="value") + assert _read_cell(None, cell, 1, 2) == { + "value": "value", "row": 1, "column": 2, + "row_span": 1, "column_span": 1, + } + + +def test_a_grid_cell_whose_span_cannot_be_read_keeps_the_default(): + cell = types.SimpleNamespace(CurrentName="value") + assert _read_cell(Pattern(CurrentRow=1), cell, 0, 0)["row_span"] == 1 + + +def test_a_grid_cell_that_is_not_there_is_none(backend, found): + _with(found, GRID, "IUIAutomationGridPattern", _grid(1, 1, {})) + assert backend.get_grid_cell(0, 0, name="Grid") is None + + +def test_a_grid_cell_the_provider_refuses_is_none(backend, found): + _with(found, GRID, "IUIAutomationGridPattern", + _grid(1, 1, item_error=OSError("gone"))) + assert backend.get_grid_cell(0, 0, name="Grid") is None + + +def test_a_control_that_is_not_a_grid_has_no_cell(backend, found): + found["set"](_control()) + assert backend.get_grid_cell(0, 0, name="Grid") is None + + +# --- selection and views ------------------------------------------------------ + +def test_a_selection_reports_its_items_and_its_rules(backend, found): + pattern = Pattern(CurrentCanSelectMultiple=True, + CurrentIsSelectionRequired=False) + pattern.GetCurrentSelection = lambda: ElementArray( + [types.SimpleNamespace(CurrentName="one")]) + _with(found, SELECTION, "IUIAutomationSelectionPattern", pattern) + assert backend.get_selection(name="List") == { + "items": ["one"], "can_select_multiple": True, "is_required": False, + } + + +def test_a_selection_the_provider_refuses_is_none(backend, found): + pattern = Pattern() + pattern.errors["GetCurrentSelection"] = OSError("gone") + _with(found, SELECTION, "IUIAutomationSelectionPattern", pattern) + assert backend.get_selection(name="List") is None + + +def test_a_control_with_no_selection_pattern_has_no_selection(backend, found): + found["set"](_control()) + assert backend.get_selection(name="List") is None + + +def _views(names, current=0): + pattern = Pattern(CurrentCurrentView=current) + pattern.GetCurrentSupportedViews = lambda: list(range(len(names))) + pattern.GetViewName = lambda view_id: names[int(view_id)] + return pattern + + +def test_the_views_of_a_control_are_listed_by_name(backend, found): + _with(found, MULTIPLE_VIEW, "IUIAutomationMultipleViewPattern", + _views(["Icons", "Details"], current=1)) + assert backend.list_views(name="Files") == { + "current": "Details", "views": ["Icons", "Details"], + } + + +def test_views_the_provider_refuses_read_as_none(backend, found): + pattern = Pattern(CurrentCurrentView=0) + pattern.errors["GetCurrentSupportedViews"] = OSError("gone") + _with(found, MULTIPLE_VIEW, "IUIAutomationMultipleViewPattern", pattern) + assert backend.list_views(name="Files") is None + + +def test_a_control_with_no_views_has_none(backend, found): + found["set"](_control()) + assert backend.list_views(name="Files") is None + + +def test_a_view_name_that_cannot_be_read_is_empty(): + assert _view_name(Pattern(), "not a number") == "" + + +def test_setting_a_view_matches_it_by_name(backend, found): + pattern = _views(["Icons", "Details"]) + _with(found, MULTIPLE_VIEW, "IUIAutomationMultipleViewPattern", pattern) + assert backend.set_view("Details", name="Files") is True + assert ("SetCurrentView", (1,)) in pattern.calls + + +def test_setting_a_view_that_does_not_exist_fails(backend, found): + _with(found, MULTIPLE_VIEW, "IUIAutomationMultipleViewPattern", + _views(["Icons"])) + assert backend.set_view("Details", name="Files") is False + + +def test_setting_a_view_the_provider_refuses_fails(backend, found): + pattern = Pattern() + pattern.errors["GetCurrentSupportedViews"] = OSError("gone") + _with(found, MULTIPLE_VIEW, "IUIAutomationMultipleViewPattern", pattern) + assert backend.set_view("Details", name="Files") is False + + +def test_setting_a_view_on_a_control_with_none_fails(backend, found): + found["set"](_control()) + assert backend.set_view("Details", name="Files") is False + + +# --- the MSAA bridge ---------------------------------------------------------- + +def test_the_legacy_fields_are_read_into_plain_values(backend, found): + _with(found, LEGACY, "IUIAutomationLegacyIAccessiblePattern", + Pattern(CurrentName="OK", CurrentValue="", CurrentDescription="d", + CurrentDefaultAction="Press", CurrentRole=43, + CurrentState=1048576)) + assert backend.legacy_info(name="OK") == { + "name": "OK", "value": "", "description": "d", + "default_action": "Press", "role": 43, "state": 1048576, + } + + +def test_a_legacy_field_the_provider_will_not_answer_reads_as_none(): + assert _read_legacy(Pattern())["name"] is None + + +def test_a_control_with_no_legacy_bridge_has_no_legacy_info(backend, found): + found["set"](_control()) + assert backend.legacy_info(name="OK") is None + + +def test_the_legacy_default_action_is_performed(backend, found): + pattern = _with(found, LEGACY, "IUIAutomationLegacyIAccessiblePattern") + assert backend.legacy_default_action(name="OK") is True + assert pattern.calls == [("DoDefaultAction", ())] + + +# --- virtualized items -------------------------------------------------------- + +def _container(item, by_property=None, find_error=None): + pattern = Pattern() + + def _find(scope, property_id, value): + if find_error is not None: + raise find_error + if by_property is not None and property_id != by_property: + return None + return item + + pattern.FindItemByProperty = _find + return pattern + + +def test_a_virtual_item_is_found_realized_and_converted(backend, found): + realize = Pattern() + item = RawElement(name="Row 500", rect=Rect(0, 0, 10, 10), + patterns={VIRTUALIZED: Unknown( + "IUIAutomationVirtualizedItemPattern", realize)}) + _with(found, ITEM_CONTAINER, "IUIAutomationItemContainerPattern", + _container(item)) + element = backend.find_virtual_item("Row 500", container_name="List") + assert element.name == "Row 500" + assert realize.calls == [("Realize", ())], ( + "a virtualized row is not a real element until it is realized" + ) + + +def test_a_virtual_item_can_be_looked_up_by_automation_id(backend, found): + item = RawElement(name="Row", rect=Rect(0, 0, 1, 1)) + _with(found, ITEM_CONTAINER, "IUIAutomationItemContainerPattern", + _container(item, by_property=backend_module._UIA_AUTOMATIONID_PROPERTY)) + assert backend.find_virtual_item("row-500", by="automation_id", + container_name="List") is not None + assert backend.find_virtual_item("row-500", container_name="List") is None + + +def test_a_virtual_item_that_is_not_in_the_container_is_none(backend, found): + _with(found, ITEM_CONTAINER, "IUIAutomationItemContainerPattern", + _container(None)) + assert backend.find_virtual_item("Row 500", container_name="List") is None + + +def test_a_container_that_refuses_the_lookup_answers_none(backend, found): + _with(found, ITEM_CONTAINER, "IUIAutomationItemContainerPattern", + _container(None, find_error=OSError("gone"))) + assert backend.find_virtual_item("Row 500", container_name="List") is None + + +def test_a_container_with_no_item_container_pattern_answers_none(backend, + found): + found["set"](_control()) + assert backend.find_virtual_item("Row 500", container_name="List") is None + + +def test_a_container_that_is_not_there_answers_none(backend, found): + assert backend.find_virtual_item("Row 500", container_name="List") is None + + +def test_an_item_that_cannot_be_realized_is_still_returned(backend, found): + realize = Pattern() + realize.errors["Realize"] = OSError("already real") + item = RawElement(name="Row", rect=Rect(0, 0, 1, 1), + patterns={VIRTUALIZED: Unknown( + "IUIAutomationVirtualizedItemPattern", realize)}) + _with(found, ITEM_CONTAINER, "IUIAutomationItemContainerPattern", + _container(item)) + assert backend.find_virtual_item("Row", container_name="List") is not None + + +def test_an_item_with_no_virtualized_pattern_is_returned_as_is(backend, + found): + item = RawElement(name="Row", rect=Rect(0, 0, 1, 1)) + _with(found, ITEM_CONTAINER, "IUIAutomationItemContainerPattern", + _container(item)) + assert backend.find_virtual_item("Row", container_name="List").name == "Row" + + +# --- text --------------------------------------------------------------------- + +class _TextRange: + def __init__(self, text="", attributes=None) -> None: + self._text = text + self._attributes = dict(attributes or {}) + self.selected = 0 + self.attribute_error = None + + def GetText(self, length): # noqa: N802 # reason: the UIA name + return self._text + + def GetAttributeValue(self, attribute_id): # noqa: N802 # UIA name + if self.attribute_error is not None: + raise self.attribute_error + return self._attributes[attribute_id] + + def Select(self): # noqa: N802 # reason: the UIA name + self.selected += 1 + + +def _text_pattern(document="", selection=None, visible=None, + find_result="missing"): + pattern = Pattern(DocumentRange=_TextRange(document)) + if find_result != "missing": + pattern.DocumentRange.FindText = lambda *args: find_result + else: + pattern.DocumentRange.FindText = lambda *args: None + pattern.GetSelection = lambda: selection + pattern.GetVisibleRanges = lambda: visible + return pattern + + +def test_the_document_text_is_read_whole(backend, found): + _with(found, TEXT, "IUIAutomationTextPattern", + _text_pattern(document="the whole document")) + assert backend.document_text(name="Editor") == "the whole document" + + +def test_a_control_with_no_text_pattern_has_no_document(backend, found): + found["set"](_control()) + assert backend.document_text(name="Editor") is None + + +def test_a_control_that_is_not_there_has_no_document(backend, found): + assert backend.document_text(name="Editor") is None + + +def test_a_document_the_provider_refuses_reads_as_none(backend, found): + pattern = Pattern() + _with(found, TEXT, "IUIAutomationTextPattern", pattern) + assert backend.document_text(name="Editor") is None + + +def test_the_selected_text_comes_from_the_first_selected_range(backend, + found): + selection = ElementArray([_TextRange("chosen")]) + _with(found, TEXT, "IUIAutomationTextPattern", + _text_pattern(selection=selection)) + assert backend.selected_text(name="Editor") == "chosen" + + +def test_an_empty_selection_reads_as_the_empty_string(backend, found): + # "" and None are different answers: nothing is selected, against there + # being no text control at all. + _with(found, TEXT, "IUIAutomationTextPattern", + _text_pattern(selection=ElementArray([]))) + assert backend.selected_text(name="Editor") == "" + + +def test_a_selection_the_provider_refuses_reads_as_none(backend, found): + _with(found, TEXT, "IUIAutomationTextPattern", + _text_pattern(selection=ElementArray(length_error=OSError("gone")))) + assert backend.selected_text(name="Editor") is None + + +@pytest.mark.parametrize("method", ["selected_text", "visible_text", + "text_attributes"]) +def test_a_control_with_no_text_pattern_has_no_text_of_any_kind(backend, + found, + method): + found["set"](_control()) + assert getattr(backend, method)(name="Editor") is None + + +def test_the_visible_text_is_the_visible_ranges_joined(backend, found): + visible = ElementArray([_TextRange("first "), _TextRange("second")]) + _with(found, TEXT, "IUIAutomationTextPattern", + _text_pattern(visible=visible)) + assert backend.visible_text(name="Editor") == "first second" + + +def test_visible_text_the_provider_refuses_reads_as_none(backend, found): + _with(found, TEXT, "IUIAutomationTextPattern", + _text_pattern(visible=ElementArray(length_error=OSError("gone")))) + assert backend.visible_text(name="Editor") is None + + +def test_finding_text_reports_whether_a_range_came_back(backend, found): + _with(found, TEXT, "IUIAutomationTextPattern", + _text_pattern(find_result=_TextRange("found"))) + assert backend.find_text("needle", name="Editor") is True + + +def test_finding_text_that_is_not_there_reports_false(backend, found): + _with(found, TEXT, "IUIAutomationTextPattern", _text_pattern()) + assert backend.find_text("needle", name="Editor") is False + + +def test_finding_text_in_a_control_that_is_not_there_reports_false(backend, + found): + assert backend.find_text("needle", name="Editor") is False + + +def test_selecting_text_selects_the_range_it_found(backend, found): + target = _TextRange("found") + _with(found, TEXT, "IUIAutomationTextPattern", + _text_pattern(find_result=target)) + assert backend.select_text("needle", name="Editor") is True + assert target.selected == 1 + + +def test_selecting_text_that_is_not_there_reports_false(backend, found): + _with(found, TEXT, "IUIAutomationTextPattern", _text_pattern()) + assert backend.select_text("needle", name="Editor") is False + + +def test_a_selection_the_control_refuses_reports_false(backend, found): + target = _TextRange("found") + + def _boom(): + raise OSError("read-only") + + target.Select = _boom + _with(found, TEXT, "IUIAutomationTextPattern", + _text_pattern(find_result=target)) + assert backend.select_text("needle", name="Editor") is False + + +def test_a_find_the_provider_refuses_reports_false(backend, found): + pattern = Pattern(DocumentRange=_TextRange()) + + def _boom(*_args): + raise OSError("gone") + + pattern.DocumentRange.FindText = _boom + _with(found, TEXT, "IUIAutomationTextPattern", pattern) + assert backend.find_text("needle", name="Editor") is False + + +_ATTRS = {40005: "Consolas", 40006: 12.0, 40007: 700, 40008: 255, 40014: True} + + +def test_text_attributes_come_from_the_selection_when_there_is_one(backend, + found): + selected = _TextRange("chosen", attributes=_ATTRS) + _with(found, TEXT, "IUIAutomationTextPattern", + _text_pattern(selection=ElementArray([selected]))) + assert backend.text_attributes(name="Editor")["font_name"] == "Consolas" + + +def test_text_attributes_fall_back_to_the_whole_document(backend, found): + pattern = _text_pattern(document="all of it", + selection=ElementArray([])) + pattern.DocumentRange._attributes = dict(_ATTRS) + _with(found, TEXT, "IUIAutomationTextPattern", pattern) + assert backend.text_attributes(name="Editor")["font_size"] == 12.0 + + +def test_a_font_weight_of_seven_hundred_is_bold(): + assert _read_text_attributes(_TextRange(attributes=_ATTRS))["bold"] is True + + +def test_a_lighter_font_weight_is_not_bold(): + attributes = {**_ATTRS, 40007: 400} + assert _read_text_attributes( + _TextRange(attributes=attributes))["bold"] is False + + +def test_an_unreadable_weight_leaves_boldness_unknown(): + # None is not False: "the control did not say" is a different answer from + # "it is not bold", and a caller may want to ask again. + text_range = _TextRange(attributes={}) + text_range.attribute_error = OSError("gone") + assert _read_text_attributes(text_range)["bold"] is None + + +def test_a_selection_that_cannot_be_read_leaves_attributes_none(backend, + found): + _with(found, TEXT, "IUIAutomationTextPattern", + _text_pattern(selection=ElementArray(length_error=OSError("gone")))) + assert backend.text_attributes(name="Editor") is None + + +def test_a_control_with_no_text_pattern_has_no_attributes(backend, found): + found["set"](_control()) + assert backend.text_attributes(name="Editor") is None + + +# --- focus -------------------------------------------------------------------- + +def test_focusing_a_control_calls_set_focus_on_it(backend, found): + raw = found["set"](_control()) + assert backend.set_focus(name="Field") is True + assert raw.focused == 1 + + +def test_focusing_a_control_that_is_not_there_fails(backend, found): + assert backend.set_focus(name="Field") is False + + +def test_a_focus_the_control_refuses_fails(backend, found): + raw = found["set"](_control()) + raw.focus_error = OSError("cannot focus") + assert backend.set_focus(name="Field") is False + + +def test_waiting_for_focus_registers_and_removes_its_handler(backend, + monkeypatch): + install_comtypes(monkeypatch) + automation = backend._automation + result = backend.wait_for_focus_change(timeout=0.01) + assert result is None, "nothing focused inside the timeout" + assert automation.focus_handlers == [], "the handler was removed again" + + +def test_waiting_for_focus_reports_the_element_that_took_it(backend, + monkeypatch): + install_comtypes(monkeypatch) + captured = {} + + def _handler_factory(sink): + captured["sink"] = sink + sink.put({"name": "Field"}) + return object() + + monkeypatch.setattr(backend, "_make_focus_handler", _handler_factory) + assert backend.wait_for_focus_change(timeout=1.0) == {"name": "Field"} + + +def test_waiting_for_focus_without_comtypes_answers_none(backend, + monkeypatch): + monkeypatch.setattr(backend, "_make_focus_handler", lambda sink: None) + assert backend.wait_for_focus_change(timeout=0.01) is None + + +def test_a_provider_that_refuses_the_subscription_answers_none(backend, + monkeypatch): + install_comtypes(monkeypatch) + backend._automation.add_error = OSError("cannot subscribe") + assert backend.wait_for_focus_change(timeout=0.01) is None + + +def test_a_handler_that_cannot_be_removed_does_not_escape(backend, + monkeypatch): + install_comtypes(monkeypatch) + backend._automation.remove_error = OSError("already gone") + assert backend.wait_for_focus_change(timeout=0.01) is None + + +def test_the_focus_handler_reports_the_element_that_was_focused(backend, + monkeypatch): + import queue + install_comtypes(monkeypatch) + sink: "queue.Queue" = queue.Queue() + handler = backend._make_focus_handler(sink) + assert handler is not None + raw = RawElement(name="Field", rect=Rect(0, 0, 1, 1)) + handler.IUIAutomationFocusChangedEventHandler_HandleFocusChangedEvent(raw) + assert sink.get_nowait()["name"] == "Field" + + +def test_a_focus_event_whose_element_is_gone_still_reports_something( + backend, monkeypatch): + import queue + install_comtypes(monkeypatch) + sink: "queue.Queue" = queue.Queue() + handler = backend._make_focus_handler(sink) + broken = RawElement(rect=Rect(0, 0, 1, 1)) + del broken.CurrentName + handler.IUIAutomationFocusChangedEventHandler_HandleFocusChangedEvent( + broken) + assert sink.get_nowait() == {"focused": True} + + +def test_no_comtypes_means_no_focus_handler(backend, monkeypatch): + monkeypatch.setitem(sys.modules, "comtypes", None) + assert backend._make_focus_handler(None) is None + + +def test_a_uia_module_without_the_event_interface_has_no_handler(backend, + monkeypatch): + install_comtypes(monkeypatch) + backend._uia_module = types.SimpleNamespace() + assert backend._make_focus_handler(None) is None + + diff --git a/test/unit_test/headless/test_accessibility_windows_patterns.py b/test/unit_test/headless/test_accessibility_windows_patterns.py new file mode 100644 index 00000000..48c53957 --- /dev/null +++ b/test/unit_test/headless/test_accessibility_windows_patterns.py @@ -0,0 +1,450 @@ +"""What the Windows backend does with a control once it has found one. + +The search that finds it is covered in `test_accessibility_windows_uia.py`; +this file starts from "here is the element" and is about the thirty-odd +control patterns hanging off it -- the half of the backend that reads a +slider's number, presses a button, expands a tree node or pulls the text out +of a document. + +They all go through one indirection, and it is where a mistake hides: + + unknown = raw.GetCurrentPattern(pattern_id) # nothing if unsupported + pattern = unknown.QueryInterface(getattr(uia_module, interface_name)) + +Two ids and one interface name per operation, none of them checked by +anything at runtime -- ask for the ValuePattern id and query the +RangeValuePattern interface and the failure is a `None` that reads exactly +like "no such control". The doubles here refuse a mismatched pair, so each +test below is also an assertion that the operation reaches for the pattern it +means. + +The other thing worth stating is the answer shape. A control that does not +support a pattern is not an error and not an empty value -- it is "no", and +each method has to spell "no" in whatever type it promised: `None` for a +read, `False` for an action, `[]` for a table. A caller cannot recover from +`False` when it meant "there is no such control", so the distinction is the +whole contract. +""" +from __future__ import annotations + +import sys + +import pytest + +from headless._uia_doubles import ( + Automation, Pattern, RawElement, Rect, UiaModule, Unknown, +) +from je_auto_control.utils.accessibility.backends import ( + windows_backend as backend_module, +) +from je_auto_control.utils.accessibility.backends.windows_backend import ( + WindowsAccessibilityBackend, +) + +VALUE = backend_module._UIA_VALUE_PATTERN_ID +INVOKE = backend_module._UIA_INVOKE_PATTERN_ID +TOGGLE = backend_module._UIA_TOGGLE_PATTERN_ID +GRID = backend_module._UIA_GRID_PATTERN_ID +GRID_ITEM = backend_module._UIA_GRIDITEM_PATTERN_ID +EXPAND = backend_module._UIA_EXPANDCOLLAPSE_PATTERN_ID +SELECTION_ITEM = backend_module._UIA_SELECTIONITEM_PATTERN_ID +RANGE = backend_module._UIA_RANGEVALUE_PATTERN_ID +SCROLL_ITEM = backend_module._UIA_SCROLLITEM_PATTERN_ID +TEXT = backend_module._UIA_TEXT_PATTERN_ID +ITEM_CONTAINER = backend_module._UIA_ITEMCONTAINER_PATTERN_ID +VIRTUALIZED = backend_module._UIA_VIRTUALIZEDITEM_PATTERN_ID +TABLE = backend_module._UIA_TABLE_PATTERN_ID +TRANSFORM = backend_module._UIA_TRANSFORM_PATTERN_ID +WINDOW = backend_module._UIA_WINDOW_PATTERN_ID +LEGACY = backend_module._UIA_LEGACYIACCESSIBLE_PATTERN_ID +SELECTION = backend_module._UIA_SELECTION_PATTERN_ID +MULTIPLE_VIEW = backend_module._UIA_MULTIPLEVIEW_PATTERN_ID + +_IS_PASSWORD = 30019 + + +@pytest.fixture(autouse=True) +def named_processes(monkeypatch): + monkeypatch.setattr(backend_module, "_process_name", + lambda pid: f"app{pid}.exe" if pid else "") + + +@pytest.fixture +def backend(monkeypatch): + monkeypatch.setattr(backend_module, "_is_available", lambda: True) + instance = WindowsAccessibilityBackend() + instance._automation = Automation() + instance._uia_module = UiaModule() + return instance + + +@pytest.fixture +def found(backend, monkeypatch): + """Answer every search with one raw element, and record what was asked.""" + state = {"raw": None, "filters": []} + + def _find_raw(name, role, app_name, automation_id, window_title=None, + contains=False): + state["filters"].append({"name": name, "role": role, + "app_name": app_name, + "automation_id": automation_id, + "window_title": window_title, + "contains": contains}) + return state["raw"] + + monkeypatch.setattr(backend, "_find_raw", _find_raw) + + def _set(raw): + state["raw"] = raw + return raw + + state["set"] = _set + return state + + +def _control(pattern_id=None, interface_name="", pattern=None, **kwargs): + """A raw element carrying at most one pattern.""" + patterns = {} + if pattern_id is not None: + patterns[pattern_id] = Unknown(interface_name, pattern or Pattern()) + kwargs.setdefault("rect", Rect(0, 0, 1, 1)) + return RawElement(patterns=patterns, **kwargs) + + +def _with(found, pattern_id, interface_name, pattern=None, **kwargs): + """Install a control carrying one pattern, and hand the pattern back.""" + pattern = pattern or Pattern() + found["set"](_control(pattern_id, interface_name, pattern, **kwargs)) + return pattern + + +# --- the pattern indirection -------------------------------------------------- + +def test_a_control_that_does_not_support_a_pattern_yields_none(backend): + assert backend._pattern(_control(), VALUE, "IUIAutomationValuePattern") is ( + None + ) + + +def test_asking_for_the_wrong_interface_yields_none_rather_than_a_pattern( + backend): + # The id and the interface name are two independent constants; a + # mismatched pair is a silent `None` that reads like "no such control". + raw = _control(VALUE, "IUIAutomationValuePattern") + assert backend._pattern(raw, VALUE, "IUIAutomationRangeValuePattern") is ( + None + ) + + +def test_a_provider_that_fails_the_pattern_query_yields_none(backend): + raw = _control(VALUE, "IUIAutomationValuePattern") + raw.pattern_error = OSError("provider stopped responding") + assert backend._pattern(raw, VALUE, "IUIAutomationValuePattern") is None + + +def test_the_uia_catch_tuple_covers_the_providers_own_error_type(): + # comtypes reports a provider failure as COMError, which derives straight + # from Exception. Until 2026-08-24 the 37 guards in this module named + # `(OSError, AttributeError, ...)` and therefore contained none of them, + # while the two walk guards in the same file already used UIA_ERRORS. + assert TypeError in backend_module._UIA_ERRORS + if sys.platform == "win32": + from _ctypes import COMError + assert COMError in backend_module._UIA_ERRORS + assert not issubclass( + COMError, (OSError, AttributeError, ValueError, TypeError)) + + +@pytest.mark.skipif(sys.platform != "win32", + reason="COMError only exists where comtypes can") +def test_a_window_that_closes_between_the_search_and_the_read_is_contained( + backend, found): + # The race the guard is for: the element was found, and the application + # owning it went away before its value could be read. The answer is "no + # value", not an exception past the executor's containment boundary. + from _ctypes import COMError + raw = _control(VALUE, "IUIAutomationValuePattern", + properties={_IS_PASSWORD: False}) + raw.pattern_error = COMError(-2147220991, "the window closed", None) + found["set"](raw) + assert backend.get_value(name="Field") is None + + +# --- value -------------------------------------------------------------------- + +def test_a_value_is_read_from_the_value_pattern(backend, found): + _with(found, VALUE, "IUIAutomationValuePattern", + Pattern(CurrentValue="typed"), properties={_IS_PASSWORD: False}) + assert backend.get_value(name="Field") == "typed" + + +def test_a_password_fields_value_is_never_handed_back(backend, found): + # UIA is supposed to mask it, but that is a convention a custom-drawn + # control can ignore -- and callers log and forward what they read. + _with(found, VALUE, "IUIAutomationValuePattern", + Pattern(CurrentValue="hunter2"), properties={_IS_PASSWORD: True}) + assert backend.get_value(name="Password") is None + + +def test_a_value_read_forwards_the_scope_it_was_given(backend, found): + _with(found, VALUE, "IUIAutomationValuePattern", + Pattern(CurrentValue="typed"), properties={_IS_PASSWORD: False}) + backend.get_value(name="Field", window_title="Editor", contains=True) + assert found["filters"][-1]["window_title"] == "Editor" + assert found["filters"][-1]["contains"] is True + + +def test_a_control_with_no_value_pattern_has_no_value(backend, found): + found["set"](_control(properties={_IS_PASSWORD: False})) + assert backend.get_value(name="Field") is None + + +def test_a_value_the_provider_will_not_answer_reads_as_none(backend, found): + _with(found, VALUE, "IUIAutomationValuePattern", Pattern(), + properties={_IS_PASSWORD: False}) + assert backend.get_value(name="Field") is None + + +def test_an_empty_value_reads_as_the_empty_string(backend, found): + _with(found, VALUE, "IUIAutomationValuePattern", + Pattern(CurrentValue=None), properties={_IS_PASSWORD: False}) + assert backend.get_value(name="Field") == "" + + +def test_a_control_that_is_not_there_has_no_value(backend, found): + assert backend.get_value(name="Nothing") is None + + +def test_setting_a_value_writes_it_as_text(backend, found): + pattern = _with(found, VALUE, "IUIAutomationValuePattern") + assert backend.set_value(42, name="Field") is True + assert pattern.calls == [("SetValue", ("42",))] + + +def test_setting_a_value_on_a_control_that_is_not_there_fails(backend, found): + assert backend.set_value("x", name="Nothing") is False + + +def test_setting_a_value_the_control_refuses_fails(backend, found): + pattern = _with(found, VALUE, "IUIAutomationValuePattern") + pattern.errors["SetValue"] = OSError("read-only") + assert backend.set_value("x", name="Field") is False + + +# --- invoke and toggle -------------------------------------------------------- + +def test_invoking_presses_the_control(backend, found): + pattern = _with(found, INVOKE, "IUIAutomationInvokePattern") + assert backend.invoke(name="OK") is True + assert pattern.calls == [("Invoke", ())] + + +def test_invoking_a_control_with_no_invoke_pattern_fails(backend, found): + found["set"](_control()) + assert backend.invoke(name="OK") is False + + +def test_invoking_a_control_that_is_not_there_fails(backend, found): + assert backend.invoke(name="OK") is False + + +def test_an_invoke_the_control_refuses_fails(backend, found): + pattern = _with(found, INVOKE, "IUIAutomationInvokePattern") + pattern.errors["Invoke"] = OSError("disabled") + assert backend.invoke(name="OK") is False + + +def test_toggling_flips_the_control(backend, found): + pattern = _with(found, TOGGLE, "IUIAutomationTogglePattern") + assert backend.toggle(name="Enabled") is True + assert pattern.calls == [("Toggle", ())] + + +def test_toggling_a_control_that_cannot_be_toggled_fails(backend, found): + found["set"](_control()) + assert backend.toggle(name="Enabled") is False + + +def test_a_toggle_the_control_refuses_fails(backend, found): + pattern = _with(found, TOGGLE, "IUIAutomationTogglePattern") + pattern.errors["Toggle"] = OSError("disabled") + assert backend.toggle(name="Enabled") is False + + +# --- expand / collapse / select / scroll -------------------------------------- + +@pytest.mark.parametrize("method,expected", [ + ("expand", "Expand"), ("collapse", "Collapse"), +]) +def test_expanding_and_collapsing_reach_the_same_pattern(backend, found, + method, expected): + pattern = _with(found, EXPAND, "IUIAutomationExpandCollapsePattern") + assert getattr(backend, method)(name="Node") is True + assert pattern.calls == [(expected, ())] + + +def test_an_action_on_a_control_without_that_pattern_fails(backend, found): + # `expand`, `select_item`, `scroll_into_view`, `move_element`, + # `resize_element`, `set_range_value`, `set_window_state` and + # `legacy_default_action` all funnel through one helper; this is that + # helper's "the control cannot do it" answer. + found["set"](_control()) + assert backend.expand(name="Node") is False + + +def test_an_action_the_control_refuses_fails(backend, found): + pattern = _with(found, EXPAND, "IUIAutomationExpandCollapsePattern") + pattern.errors["Expand"] = OSError("already expanded") + assert backend.expand(name="Node") is False + + +def test_an_action_on_a_control_that_is_not_there_fails(backend, found): + assert backend.expand(name="Node") is False + + +@pytest.mark.parametrize("code,expected", [ + (0, "collapsed"), (1, "expanded"), (2, "partial"), (3, "leaf"), +]) +def test_an_expand_state_is_reported_by_name(backend, found, code, expected): + _with(found, EXPAND, "IUIAutomationExpandCollapsePattern", + Pattern(CurrentExpandCollapseState=code)) + assert backend.expand_state(name="Node") == expected + + +def test_an_unknown_expand_state_reads_as_none(backend, found): + _with(found, EXPAND, "IUIAutomationExpandCollapsePattern", + Pattern(CurrentExpandCollapseState=9)) + assert backend.expand_state(name="Node") is None + + +def test_an_unreadable_expand_state_reads_as_none(backend, found): + _with(found, EXPAND, "IUIAutomationExpandCollapsePattern", + Pattern(CurrentExpandCollapseState="sideways")) + assert backend.expand_state(name="Node") is None + + +def test_a_control_that_does_not_expand_has_no_expand_state(backend, found): + found["set"](_control()) + assert backend.expand_state(name="Node") is None + + +def test_a_control_that_is_not_there_has_no_expand_state(backend, found): + assert backend.expand_state(name="Node") is None + + +def test_selecting_an_item_reaches_the_selection_item_pattern(backend, found): + pattern = _with(found, SELECTION_ITEM, + "IUIAutomationSelectionItemPattern") + assert backend.select_item(name="Row") is True + assert pattern.calls == [("Select", ())] + + +def test_scrolling_into_view_reaches_the_scroll_item_pattern(backend, found): + pattern = _with(found, SCROLL_ITEM, "IUIAutomationScrollItemPattern") + assert backend.scroll_into_view(name="Row") is True + assert pattern.calls == [("ScrollIntoView", ())] + + +# --- range -------------------------------------------------------------------- + +def test_a_range_is_read_as_three_floats(backend, found): + _with(found, RANGE, "IUIAutomationRangeValuePattern", + Pattern(CurrentValue=3, CurrentMinimum=0, CurrentMaximum=10)) + assert backend.get_range(name="Volume") == { + "value": 3.0, "minimum": 0.0, "maximum": 10.0, + } + + +def test_a_range_that_cannot_be_read_is_none(backend, found): + _with(found, RANGE, "IUIAutomationRangeValuePattern", + Pattern(CurrentValue="loud", CurrentMinimum=0, CurrentMaximum=10)) + assert backend.get_range(name="Volume") is None + + +def test_a_control_with_no_range_has_none(backend, found): + found["set"](_control()) + assert backend.get_range(name="Volume") is None + + +def test_a_control_that_is_not_there_has_no_range(backend, found): + assert backend.get_range(name="Volume") is None + + +def test_setting_a_range_value_writes_it_as_a_float(backend, found): + pattern = _with(found, RANGE, "IUIAutomationRangeValuePattern") + assert backend.set_range_value(7, name="Volume") is True + assert pattern.calls == [("SetValue", (7.0,))] + + +# --- transform and window state ----------------------------------------------- + +def test_moving_a_control_writes_floats(backend, found): + pattern = _with(found, TRANSFORM, "IUIAutomationTransformPattern") + assert backend.move_element(10, 20, name="Panel") is True + assert pattern.calls == [("Move", (10.0, 20.0))] + + +def test_resizing_a_control_writes_floats(backend, found): + pattern = _with(found, TRANSFORM, "IUIAutomationTransformPattern") + assert backend.resize_element(300, 400, name="Panel") is True + assert pattern.calls == [("Resize", (300.0, 400.0))] + + +@pytest.mark.parametrize("state,code", [ + ("normal", 0), ("maximized", 1), ("minimized", 2), ("MAXIMIZED", 1), +]) +def test_a_window_state_is_written_as_its_visual_state_code(backend, found, + state, code): + pattern = _with(found, WINDOW, "IUIAutomationWindowPattern") + assert backend.set_window_state(state, name="Editor") is True + assert pattern.calls == [("SetWindowVisualState", (code,))] + + +def test_an_unknown_window_state_is_refused_before_any_search(backend, found): + assert backend.set_window_state("shaded", name="Editor") is False + assert found["filters"] == [], "it never went looking" + + +@pytest.mark.parametrize("code,expected", [ + (0, "running"), (1, "closing"), (2, "ready"), (3, "blocked_by_modal"), + (4, "not_responding"), +]) +def test_a_window_interaction_state_is_reported_by_name(backend, found, code, + expected): + _with(found, WINDOW, "IUIAutomationWindowPattern", + Pattern(CurrentWindowInteractionState=code)) + assert backend.window_interaction_state(name="Editor") == expected + + +def test_an_unknown_interaction_state_reads_as_none(backend, found): + _with(found, WINDOW, "IUIAutomationWindowPattern", + Pattern(CurrentWindowInteractionState=99)) + assert backend.window_interaction_state(name="Editor") is None + + +def test_an_unreadable_interaction_state_reads_as_none(backend, found): + _with(found, WINDOW, "IUIAutomationWindowPattern", Pattern()) + assert backend.window_interaction_state(name="Editor") is None + + +def test_a_control_that_is_not_a_window_has_no_interaction_state(backend, + found): + found["set"](_control()) + assert backend.window_interaction_state(name="Editor") is None + + +def test_a_control_that_is_not_there_has_no_interaction_state(backend, found): + assert backend.window_interaction_state(name="Editor") is None + + +# --- state -------------------------------------------------------------------- + +def test_the_state_of_a_control_is_read_from_it(backend, found): + found["set"](_control(properties={_IS_PASSWORD: False, + 30029: True, 30045: "typed", + 30046: False})) + assert backend.get_state(name="Field")["value"] == "typed" + + +def test_the_state_of_a_control_that_is_not_there_is_none(backend, found): + assert backend.get_state(name="Field") is None diff --git a/test/unit_test/headless/test_accessibility_windows_query.py b/test/unit_test/headless/test_accessibility_windows_query.py new file mode 100644 index 00000000..b860a374 --- /dev/null +++ b/test/unit_test/headless/test_accessibility_windows_query.py @@ -0,0 +1,447 @@ +"""Where a UIA search starts, and what a control is actually holding. + +`windows_query.py` and `windows_state.py` are the two halves of the Windows +accessibility backend that were split out of it, and both sat near 20% on +every square -- including the Windows ones, because a test that reaches them +needs a UIAutomation provider and a desktop with windows on it. + +Neither needs either. Both take the automation object as an argument, and +every COM import in them is lazy, so a recorder standing in for UIA drives +them from all nine squares. + +The two files encode measurements, and the tests below are what stops those +being quietly undone: + +* **A desktop-rooted `FindAll` is one call that cannot be stopped.** Measured + at ~61 s for 2,085 elements, against 0.14 s starting from one window. So + `search_roots` yields *windows*, front-most first, and `walk_elements` + walks node by node -- asking for 200 elements has to cost 200 elements' + worth of work no matter how large the window is. +* **Properties are cross-process reads.** They come back through a cache + request built once per walk; reading them individually is thousands of + round trips. +* **comtypes returns a wrapper around a NULL pointer, not `None`.** So + `child is not None` is *true* at the end of a sibling list, and the walk + collects a phantom element that raises the moment anything reads it. + Truthiness is the check that works, and a stub that returned `None` would + never catch the difference -- so the one here returns a false-y wrapper, + exactly as comtypes does. +* **An unsupported pattern answers with a default**, not an error: an empty + string for a control that has no value at all. Reading it without asking + whether the pattern exists turns "no such concept" into "it is empty", + which is the more misleading of the two because a caller acts on it. +* **A password field's contents never leave.** UIA is supposed to mask them, + but that is a convention a custom-drawn control can ignore. +""" +from __future__ import annotations + +import sys +import types + +import pytest + +from je_auto_control.utils.accessibility.backends import windows_query as query +from je_auto_control.utils.accessibility.backends.windows_query import ( + CACHED_PROPERTIES, _is_null, search_root, search_roots, walk_elements, +) +from je_auto_control.utils.accessibility.backends.windows_state import ( + TOGGLE_STATES, is_password, read_state, +) +from je_auto_control.utils.accessibility.element import ( + AccessibilityNotAvailableError, +) + + +class _NullPointer: + """What comtypes hands back for "no such element": false-y, not None.""" + + def __bool__(self) -> bool: + return False + + +class _Element: + """A UIA element: a name, some children, and scripted property reads.""" + + def __init__(self, name: str = "", children=None, properties=None, + error=None) -> None: + self.name = name + self.children = list(children or []) + self.properties = dict(properties or {}) + self.error = error + self.reads = [] + + def GetCurrentPropertyValue(self, property_id): # noqa: N802 # UIA name + self.reads.append(property_id) + if self.error is not None: + raise self.error + return self.properties.get(property_id) + + def __repr__(self) -> str: # pragma: no cover - debugging aid + return f"<{self.name}>" + + +class _CacheRequest: + def __init__(self) -> None: + self.properties = [] + + def AddProperty(self, property_id): # noqa: N802 # reason: UIA name + self.properties.append(property_id) + + +class _Walker: + """The control-view walker, over `_Element.children`.""" + + def __init__(self, world) -> None: + self._world = world + + def GetFirstChildElementBuildCache(self, node, request): # noqa: N802 + self._world.cache_requests.append(request) + if node in self._world.first_child_errors: + raise self._world.first_child_errors[node] + return node.children[0] if node.children else _NullPointer() + + def GetNextSiblingElementBuildCache(self, node, request): # noqa: N802 + if node in self._world.sibling_errors: + raise self._world.sibling_errors[node] + for parent in self._world.parents_of(node): + index = parent.children.index(node) + if index + 1 < len(parent.children): + return parent.children[index + 1] + return _NullPointer() + + +class _Automation: + def __init__(self, root=None, handles=None) -> None: + self.root = root or _Element("desktop") + self.handles = dict(handles or {}) # hwnd -> element (or exception) + self.cache_requests = [] + self.first_child_errors = {} + self.sibling_errors = {} + self.ControlViewWalker = _Walker(self) # noqa: N815 # reason: UIA + + def GetRootElement(self): # noqa: N802 # reason: the UIA name + return self.root + + def CreateCacheRequest(self): # noqa: N802 # reason: the UIA name + return _CacheRequest() + + def ElementFromHandle(self, hwnd): # noqa: N802 # reason: the UIA name + found = self.handles.get(hwnd) + if isinstance(found, Exception): + raise found + return found + + def parents_of(self, node): + stack = [self.root, *[h for h in self.handles.values() + if isinstance(h, _Element)]] + seen = set() + while stack: + current = stack.pop() + if id(current) in seen: + continue + seen.add(id(current)) + if node in current.children: + yield current + stack.extend(current.children) + + +@pytest.fixture +def windows(monkeypatch): + """Stand in for `get_all_window_hwnd`, on every platform.""" + listing = [] + + def _get_all_window_hwnd(): + return list(listing) + + module = types.ModuleType( + "je_auto_control.windows.window.windows_window_manage") + module.get_all_window_hwnd = _get_all_window_hwnd + package = types.ModuleType("je_auto_control.windows.window") + package.windows_window_manage = module + monkeypatch.setitem(sys.modules, "je_auto_control.windows.window", package) + monkeypatch.setitem( + sys.modules, "je_auto_control.windows.window.windows_window_manage", + module) + return listing + + +# --- choosing a root ---------------------------------------------------------- + +def test_an_unscoped_search_starts_at_the_desktop(): + automation = _Automation() + assert search_root(automation, None) is automation.root + + +def test_a_window_title_is_matched_as_a_case_insensitive_substring(windows): + target = _Element("notepad") + automation = _Automation(handles={101: target}) + windows.extend([(100, "Calculator"), (101, "Untitled - Notepad")]) + assert search_root(automation, " notePAD ") is target + + +def test_the_first_window_whose_title_matches_wins(windows): + first, second = _Element("first"), _Element("second") + automation = _Automation(handles={1: first, 2: second}) + windows.extend([(1, "Report - Editor"), (2, "Notes - Editor")]) + assert search_root(automation, "Editor") is first + + +def test_a_title_that_matches_nothing_says_so(windows): + windows.append((1, "Calculator")) + with pytest.raises(AccessibilityNotAvailableError, match="Notepad"): + search_root(_Automation(), "Notepad") + + +def test_a_window_with_no_title_is_not_a_match(windows): + windows.append((1, None)) + with pytest.raises(AccessibilityNotAvailableError): + search_root(_Automation(), "anything") + + +def test_a_window_uia_will_not_open_is_skipped(windows): + later = _Element("later") + automation = _Automation(handles={1: None, 2: later}) + windows.extend([(1, "Editor one"), (2, "Editor two")]) + assert search_root(automation, "Editor") is later + + +# --- iterating the roots ------------------------------------------------------ + +def test_an_unscoped_walk_yields_windows_rather_than_the_desktop(windows): + # Same coverage as a desktop-rooted walk, but the caller can stop. That + # is the 0.22 s / 61 s difference the module's docstring measured. + first, second = _Element("one"), _Element("two") + automation = _Automation(handles={1: first, 2: second}) + windows.extend([(1, "one"), (2, "two")]) + assert list(search_roots(automation, None)) == [first, second] + + +def test_the_roots_come_back_in_z_order(windows): + # EnumWindows returns front-to-back, so the window the user is actually + # looking at is searched first. + front, back = _Element("front"), _Element("back") + automation = _Automation(handles={9: front, 1: back}) + windows.extend([(9, "front"), (1, "back")]) + assert [e.name for e in search_roots(automation, None)] == ["front", + "back"] + + +def test_a_window_that_closes_between_enumerate_and_use_is_skipped(windows): + survivor = _Element("survivor") + automation = _Automation(handles={1: OSError("window closed"), + 2: survivor}) + windows.extend([(1, "gone"), (2, "survivor")]) + assert list(search_roots(automation, None)) == [survivor] + + +def test_a_null_pointer_root_is_skipped(windows): + # comtypes answers with a wrapper around NULL, which is not None. + survivor = _Element("survivor") + automation = _Automation(handles={1: _NullPointer(), 2: survivor}) + windows.extend([(1, "phantom"), (2, "survivor")]) + assert list(search_roots(automation, None)) == [survivor] + + +def test_a_scoped_walk_yields_exactly_one_root(windows): + target = _Element("notepad") + automation = _Automation(handles={1: target}) + windows.append((1, "Untitled - Notepad")) + assert list(search_roots(automation, "Notepad")) == [target] + + +@pytest.mark.parametrize("value,expected", [ + (None, True), (_NullPointer(), True), (_Element("real"), False), +]) +def test_null_detection_uses_truthiness_not_identity(value, expected): + assert _is_null(value) is expected + + +# --- walking ------------------------------------------------------------------ + +def _tree(): + leaf_a = _Element("a") + leaf_b = _Element("b") + group = _Element("group", children=[leaf_a, leaf_b]) + tail = _Element("tail") + root = _Element("root", children=[group, tail]) + return root, [group, leaf_a, leaf_b, tail] + + +def test_the_walk_is_depth_first_in_reading_order(): + root, expected = _tree() + automation = _Automation(root=root) + walked = list(walk_elements(automation, root, 10)) + assert [e.name for e in walked] == [e.name for e in expected] + + +def test_the_walk_stops_at_the_limit(): + root, _expected = _tree() + automation = _Automation(root=root) + assert len(list(walk_elements(automation, root, 2))) == 2 + + +@pytest.mark.parametrize("limit", [0, -1]) +def test_a_walk_with_no_budget_asks_for_nothing(limit): + root, _expected = _tree() + automation = _Automation(root=root) + assert list(walk_elements(automation, root, limit)) == [] + assert automation.cache_requests == [], "not one cross-process call" + + +def test_every_property_the_conversion_reads_is_cached_up_front(): + # Each `Current*` read is another cross-process call; the bulk request is + # what makes converting 500 elements cost 0.02 s. + root, _expected = _tree() + automation = _Automation(root=root) + list(walk_elements(automation, root, 10)) + assert automation.cache_requests + assert automation.cache_requests[0].properties == list(CACHED_PROPERTIES) + + +def test_a_node_that_refuses_its_first_child_ends_that_branch(): + root, _expected = _tree() + automation = _Automation(root=root) + automation.first_child_errors[root.children[0]] = OSError("gone") + walked = [e.name for e in walk_elements(automation, root, 10)] + assert walked == ["group", "tail"], "the sibling after it still came back" + + +def test_a_node_that_refuses_a_sibling_keeps_what_it_already_had(): + root, _expected = _tree() + automation = _Automation(root=root) + automation.sibling_errors[root.children[0]] = OSError("gone") + walked = [e.name for e in walk_elements(automation, root, 10)] + assert walked == ["group", "a", "b"], "tail was never reached" + + +def test_a_node_with_more_children_than_the_budget_is_capped(): + children = [_Element(f"c{index}") for index in range(50)] + root = _Element("root", children=children) + automation = _Automation(root=root) + assert len(list(walk_elements(automation, root, 3))) == 3 + + +# --- reading a control's state ------------------------------------------------ + +_IS_PASSWORD = 30019 +_IS_VALUE_AVAILABLE, _VALUE = 30029, 30045 +_IS_TOGGLE_AVAILABLE, _TOGGLE = 30041, 30086 +_IS_SELECTION_AVAILABLE, _SELECTED = 30036, 30079 +_IS_RANGE_AVAILABLE, _RANGE = 30034, 30047 +_READONLY = 30046 +_LEGACY_VALUE = 30093 + + +def _control(properties): + """A UIA element whose property reads answer from a plain dict.""" + return _Element(properties=properties) + + +def test_a_password_field_reports_only_that_it_is_one(): + state = read_state(_control({_IS_PASSWORD: True, + _IS_VALUE_AVAILABLE: True, + _VALUE: "hunter2"})) + assert state == {"password": True} + assert "hunter2" not in str(state) + + +def test_an_element_whose_password_flag_cannot_be_read_counts_as_one(): + # Failing closed: the cost of being wrong the other way is a credential. + element = _control({_IS_VALUE_AVAILABLE: True, _VALUE: "secret"}) + element.error = OSError("provider gone") + assert is_password(element) is True + assert read_state(element) == {"password": True} + + +def test_an_ordinary_field_is_not_a_password(): + assert is_password(_control({_IS_PASSWORD: False})) is False + + +def test_a_value_is_read_only_when_the_pattern_is_supported(): + # An unsupported pattern answers with the default -- an empty string -- + # which reads as "the value is empty" rather than "there is no value". + absent = _control({_IS_PASSWORD: False, _VALUE: ""}) + assert "value" not in read_state(absent) + + +def test_a_supported_value_comes_back_as_text_with_its_read_only_flag(): + state = read_state(_control({_IS_PASSWORD: False, + _IS_VALUE_AVAILABLE: True, + _VALUE: 42, _READONLY: True})) + assert state["value"] == "42" + assert state["read_only"] is True + + +def test_an_empty_supported_value_is_still_a_value(): + state = read_state(_control({_IS_PASSWORD: False, + _IS_VALUE_AVAILABLE: True, _VALUE: None})) + assert state.get("value") is None, "None never reached the state at all" + + +def test_a_control_with_no_value_pattern_falls_back_to_the_legacy_one(): + # Win32 controls with no UIA provider still answer the legacy accessible + # interface, and that is the only value they will ever report. + state = read_state(_control({_IS_PASSWORD: False, + _LEGACY_VALUE: "legacy text"})) + assert state["value"] == "legacy text" + assert "read_only" not in state + + +def test_an_empty_legacy_value_is_not_reported(): + state = read_state(_control({_IS_PASSWORD: False, _LEGACY_VALUE: ""})) + assert "value" not in state + + +@pytest.mark.parametrize("code,expected", sorted(TOGGLE_STATES.items())) +def test_a_toggle_state_is_reported_by_name(code, expected): + state = read_state(_control({_IS_PASSWORD: False, + _IS_TOGGLE_AVAILABLE: True, + _TOGGLE: code})) + assert state["toggle"] == expected + + +def test_an_unknown_toggle_code_is_reported_as_itself(): + state = read_state(_control({_IS_PASSWORD: False, + _IS_TOGGLE_AVAILABLE: True, _TOGGLE: 9})) + assert state["toggle"] == "9" + + +def test_a_selection_state_is_reported_as_a_bool(): + state = read_state(_control({_IS_PASSWORD: False, + _IS_SELECTION_AVAILABLE: True, + _SELECTED: 1})) + assert state["selected"] is True + + +def test_a_range_value_is_reported_as_a_float(): + state = read_state(_control({_IS_PASSWORD: False, + _IS_RANGE_AVAILABLE: True, _RANGE: 3})) + assert state["number"] == 3.0 + assert isinstance(state["number"], float) + + +def test_a_control_that_supports_nothing_reports_nothing(): + assert read_state(_control({_IS_PASSWORD: False})) == {} + + +def test_a_control_that_supports_several_patterns_reports_all_of_them(): + state = read_state(_control({ + _IS_PASSWORD: False, + _IS_VALUE_AVAILABLE: True, _VALUE: "7", _READONLY: False, + _IS_TOGGLE_AVAILABLE: True, _TOGGLE: 1, + _IS_SELECTION_AVAILABLE: True, _SELECTED: True, + _IS_RANGE_AVAILABLE: True, _RANGE: 7.5, + })) + assert state == {"value": "7", "read_only": False, "toggle": "on", + "selected": True, "number": 7.5} + + +def test_the_uia_error_tuple_contains_the_com_error_on_windows(): + # comtypes reports provider failures as COMError, which inherits from + # Exception and from none of the usual suspects -- so an + # `except (OSError, AttributeError)` around a UIA call does not contain + # it, and a window closing mid-walk surfaces exactly that way. + assert OSError in query.UIA_ERRORS + if sys.platform == "win32": + from _ctypes import COMError + assert COMError in query.UIA_ERRORS diff --git a/test/unit_test/headless/test_accessibility_windows_uia.py b/test/unit_test/headless/test_accessibility_windows_uia.py new file mode 100644 index 00000000..6ebaa8dd --- /dev/null +++ b/test/unit_test/headless/test_accessibility_windows_uia.py @@ -0,0 +1,538 @@ +"""Setting up UIAutomation, listing a desktop, and converting an element. + +`backends/windows_backend.py` is the biggest single hole in this project's +coverage. It stayed one on the Windows squares as much as the others: a test +that reaches it needs a UIAutomation provider, a desktop with windows on it, +and applications willing to answer. But every `comtypes` import in the module +is inside a function and the automation object lives on the instance, so +doubles reach all of it -- from every square. + +This file covers getting there and getting back: the automation object, the +listing, the search, and the conversion. The control patterns are in +`test_accessibility_windows_patterns.py`. + +Three decisions in here were made against measurements, and are the kind that +get undone by a well-meaning simplification: + +* **`CUIAutomation8` is asked for first.** It is the only class that hands out + `IUIAutomation2`, which is the only way to bound how long UIA waits on an + application's provider. A full-screen game that never answered made one + `ElementFromHandle` block for 60 seconds; with the timeout set, 1.0 s. Only + the *connect* step is bounded -- how long a legitimate query may take is a + different question. +* **The listing pulls one window at a time.** A desktop-rooted walk measured + ~61 s and cannot be interrupted once started. The budget is checked + *before* the next window is fetched, not after, because obtaining a + window's root element is itself a cross-process call that blocks against a + hung application -- fetching one more root only to find the results already + complete cost exactly that. +* **An `app_name` filter walks more than it keeps.** Most elements in a + window belong to that window's application, so a small overscan is plenty + -- but it is still bounded, or an unmatched filter walks an entire subtree + to return nothing. +""" +from __future__ import annotations + +import sys +import types + +import pytest + +from headless._uia_doubles import ( + Automation, RawElement, Rect, UiaModule, install_comtypes, +) +from je_auto_control.utils.accessibility.backends import ( + windows_backend as backend_module, +) +from je_auto_control.utils.accessibility.backends.windows_backend import ( + WindowsAccessibilityBackend, _convert_uia, _create_automation, + _process_name, _read_properties, _safe_name, +) +from je_auto_control.utils.accessibility.element import ( + AccessibilityNotAvailableError, +) + + +@pytest.fixture(autouse=True) +def named_processes(monkeypatch): + """Name a pid without asking Windows about a pid the test invented.""" + monkeypatch.setattr(backend_module, "_process_name", + lambda pid: f"app{pid}.exe" if pid else "") + + +@pytest.fixture +def backend(monkeypatch): + """An available backend whose automation object is already built.""" + monkeypatch.setattr(backend_module, "_is_available", lambda: True) + instance = WindowsAccessibilityBackend() + instance._automation = Automation() + instance._uia_module = UiaModule() + return instance + + +def _element(name="", control_type=50000, rect=None, process_id=0, + automation_id="", enabled=True, cached=False): + return RawElement(name=name, control_type=control_type, + rect=rect or Rect(0, 0, 10, 20), process_id=process_id, + automation_id=automation_id, enabled=enabled, + cached=cached) + + +def _raise(error): + """Raise the error a fixture parked in its state dict. + + Raising the subscript directly reads as ``raise None`` to an analyser + that infers the slot from the dict literal declaring it, and the + ``is not None`` guard does not narrow that inference. Passing the error + through a parameter says what it is where it is raised. + """ + raise error + + +# --- availability and the automation object ----------------------------------- + +def test_a_windows_box_without_comtypes_refuses_and_says_how_to_fix_it( + monkeypatch): + monkeypatch.setattr(backend_module, "_is_available", lambda: False) + instance = WindowsAccessibilityBackend() + with pytest.raises(AccessibilityNotAvailableError, match="pip install"): + instance.list_elements() + + +def test_the_probe_reports_whether_comtypes_imports(monkeypatch): + install_comtypes(monkeypatch) + assert backend_module._is_available() is True + monkeypatch.setitem(sys.modules, "comtypes.client", None) + assert backend_module._is_available() is False + + +def test_the_backend_names_the_api_it_uses(backend): + assert backend.name == "windows-uia" + + +def test_the_automation_object_is_built_once_and_kept(monkeypatch): + monkeypatch.setattr(backend_module, "_is_available", lambda: True) + install_comtypes(monkeypatch) + instance = WindowsAccessibilityBackend() + first = instance._ensure_automation() + assert instance._ensure_automation() is first + + +def test_a_missing_uiautomation_dll_is_reported_as_unavailable(monkeypatch): + # The DLL is part of Windows, but a stripped image or a broken + # registration is a real state and must not surface as an OSError. + monkeypatch.setattr(backend_module, "_is_available", lambda: True) + install_comtypes(monkeypatch, module_error=OSError("no such module")) + instance = WindowsAccessibilityBackend() + with pytest.raises(AccessibilityNotAvailableError, + match="UIAutomationCore.dll"): + instance._ensure_automation() + + +def test_the_bounded_provider_wait_is_asked_for_first(monkeypatch): + # CUIAutomation8 is the only class that hands out IUIAutomation2, which + # is the only way to bound how long UIA waits on a provider. + created = install_comtypes(monkeypatch, uia_module=UiaModule()) + automation = _create_automation(UiaModule()) + [(clsid, interface)] = created + assert clsid == backend_module._CLSID_CUIAUTOMATION8 + assert interface == "IUIAutomation2" + assert automation.ConnectionTimeout == 1000 + + +def test_an_older_windows_falls_back_to_the_unbounded_class(monkeypatch): + created = install_comtypes(monkeypatch) + _create_automation(UiaModule(has_iuiautomation2=False)) + [(clsid, interface)] = created + assert clsid == backend_module._CLSID_CUIAUTOMATION + assert interface == "IUIAutomation" + + +def test_a_refused_iuiautomation2_falls_back_rather_than_failing(monkeypatch): + calls = [] + + def _create(clsid, interface=None): + calls.append(clsid) + if clsid == backend_module._CLSID_CUIAUTOMATION8: + raise OSError("class not registered") + return Automation() + + install_comtypes(monkeypatch, create=_create) + assert _create_automation(UiaModule()) is not None + assert calls == [backend_module._CLSID_CUIAUTOMATION8, + backend_module._CLSID_CUIAUTOMATION] + + +# --- listing ------------------------------------------------------------------ + +@pytest.fixture +def listing(monkeypatch): + """Script `search_roots` and `walk_elements` for the listing tests.""" + state = {"roots": [], "under": {}, "walk_error": None, "budgets": [], + "roots_pulled": 0} + + def _search_roots(automation, window_title): + state["window_title"] = window_title + for root in state["roots"]: + state["roots_pulled"] += 1 + yield root + + def _walk_elements(automation, root, limit): + state["budgets"].append(limit) + if state["walk_error"] is not None: + _raise(state["walk_error"]) + for element in state["under"].get(id(root), []): + yield element + + monkeypatch.setattr(backend_module, "search_roots", _search_roots) + monkeypatch.setattr(backend_module, "walk_elements", _walk_elements) + return state + + +def test_a_listing_returns_the_window_and_then_its_contents(backend, listing): + window = _element(name="Editor", process_id=7) + child = _element(name="OK", process_id=7, cached=True) + listing["roots"] = [window] + listing["under"][id(window)] = [child] + names = [e.name for e in backend.list_elements()] + assert names == ["Editor", "OK"] + + +def test_a_scoped_listing_does_not_add_the_window_itself(backend, listing): + # Searching a window's descendants does not include the window element, + # so the unscoped walk adds it back -- and the scoped one must not, or a + # caller asking about one window gets it twice. + window = _element(name="Editor", process_id=7) + child = _element(name="OK", process_id=7, cached=True) + listing["roots"] = [window] + listing["under"][id(window)] = [child] + names = [e.name for e in backend.list_elements(window_title="Editor")] + assert names == ["OK"] + assert listing["window_title"] == "Editor" + + +def test_a_listing_stops_pulling_windows_once_it_has_enough(backend, listing): + # Obtaining a window's root element is itself a cross-process call that + # blocks against a hung application, so the budget is checked before the + # next one is fetched rather than after. + first, second = _element(name="one"), _element(name="two") + listing["roots"] = [first, second] + assert len(backend.list_elements(max_results=1)) == 1 + assert listing["roots_pulled"] == 1 + + +def test_a_listing_asked_for_nothing_walks_nothing(backend, listing): + listing["roots"] = [_element(name="one")] + assert backend.list_elements(max_results=0) == [] + assert listing["roots_pulled"] == 0 + + +def test_a_negative_maximum_is_treated_as_none(backend, listing): + listing["roots"] = [_element(name="one")] + assert backend.list_elements(max_results=-5) == [] + + +def test_an_app_name_filter_keeps_only_that_application(backend, listing): + window = _element(name="Editor", process_id=7) + listing["roots"] = [window] + listing["under"][id(window)] = [ + _element(name="mine", process_id=7, cached=True), + _element(name="theirs", process_id=8, cached=True), + ] + names = [e.name for e in backend.list_elements(app_name="app7.exe")] + assert names == ["Editor", "mine"] + + +def test_an_app_name_filter_walks_further_than_it_keeps(backend, listing): + # Most elements in a window belong to that window's application, so the + # overscan is small -- but an unmatched filter must still be bounded. + window = _element(name="Editor", process_id=7) + listing["roots"] = [window] + backend.list_elements(app_name="app7.exe", max_results=10) + assert listing["budgets"][0] > 10 + assert listing["budgets"][0] < 10 * 100, "bounded, not unbounded" + + +def test_a_filtered_window_that_does_not_match_is_not_listed(backend, + listing): + listing["roots"] = [_element(name="Editor", process_id=8)] + assert backend.list_elements(app_name="app7.exe") == [] + + +def test_an_unresponsive_window_does_not_lose_the_whole_listing(backend, + listing): + window = _element(name="Editor", process_id=7) + listing["roots"] = [window] + listing["walk_error"] = OSError("provider stopped responding") + assert [e.name for e in backend.list_elements()] == ["Editor"] + + +def test_an_element_that_cannot_be_converted_is_skipped(backend, listing): + window = _element(name="Editor", process_id=7) + broken = RawElement(cached=True) + del broken.CachedName + listing["roots"] = [window] + listing["under"][id(window)] = [broken, + _element(name="OK", cached=True)] + assert [e.name for e in backend.list_elements()] == ["Editor", "OK"] + + +def test_a_desktop_with_no_windows_lists_nothing(backend, listing): + assert backend.list_elements() == [] + + +def test_a_listing_stops_mid_window_once_it_has_enough(backend, listing): + # The walk is a generator for exactly this: a window with thousands of + # nodes costs what was asked for, not what it contains. + window = _element(name="Editor") + listing["roots"] = [window] + listing["under"][id(window)] = [ + _element(name=f"c{index}", cached=True) for index in range(5) + ] + names = [e.name for e in backend.list_elements(max_results=2)] + assert names == ["Editor", "c0"] + + +# --- searching for one control ------------------------------------------------ + +@pytest.fixture +def search(monkeypatch, listing): + """The same scripting, for the `_find_raw` path.""" + return listing + + +def test_a_search_returns_the_first_match(backend, search): + window = _element(name="Editor") + wanted = _element(name="OK", cached=True) + search["roots"] = [window] + search["under"][id(window)] = [_element(name="Cancel", cached=True), + wanted] + assert backend._find_raw("OK", None, None, None) is wanted + + +def test_an_unscoped_search_can_match_the_window_itself(backend, search): + window = _element(name="Editor") + search["roots"] = [window] + assert backend._find_raw("Editor", None, None, None) is window + + +def test_a_scoped_search_does_not_match_the_window_itself(backend, search): + window = _element(name="Editor") + search["roots"] = [window] + assert backend._find_raw("Editor", None, None, None, + window_title="Editor") is None + + +def test_a_search_can_match_by_automation_id(backend, search): + window = _element(name="Editor") + wanted = _element(name="", automation_id="okButton", cached=True) + search["roots"] = [window] + search["under"][id(window)] = [wanted] + assert backend._find_raw(None, None, None, "okButton") is wanted + + +def test_a_search_by_automation_id_ignores_a_different_one(backend, search): + window = _element(name="Editor") + search["roots"] = [window] + search["under"][id(window)] = [ + _element(name="OK", automation_id="other", cached=True), + ] + assert backend._find_raw("OK", None, None, "okButton") is None + + +def test_a_search_can_match_a_substring(backend, search): + window = _element(name="Window") + wanted = _element(name="OK and close", cached=True) + search["roots"] = [window] + search["under"][id(window)] = [wanted] + assert backend._find_raw("ok and", None, None, None, + contains=True) is wanted + assert backend._find_raw("ok and", None, None, None) is None + + +def test_a_scoped_search_is_allowed_to_go_deeper(backend, search): + # Naming a window says "it is in here, find it"; not naming one says + # "look around", and stays cheap. A browser window holds thousands of + # nodes and a real target can sit well past any small cap. + window = _element(name="Editor") + search["roots"] = [window] + backend._find_raw("nothing", None, None, None) + unscoped = search["budgets"][0] + search["budgets"].clear() + backend._find_raw("nothing", None, None, None, window_title="Editor") + assert search["budgets"][0] > unscoped + + +def test_a_search_that_fails_on_the_provider_answers_none(backend, search): + search["roots"] = [_element(name="Editor")] + search["walk_error"] = OSError("provider stopped responding") + assert backend._find_raw("OK", None, None, None) is None + + +def test_a_search_of_an_empty_desktop_answers_none(backend, search): + assert backend._find_raw("OK", None, None, None) is None + + +def test_a_search_gives_up_rather_than_walking_the_whole_desktop(backend, + search): + # Unbounded, a target that is not there walks every window on the + # desktop and costs about a minute to answer "no". + first, second = _element(name="one"), _element(name="two") + search["roots"] = [first, second] + search["under"][id(first)] = [ + _element(name=f"c{index}", cached=True) + for index in range(backend_module._FIND_SCAN_LIMIT + 1) + ] + assert backend._find_raw("nothing", None, None, None) is None + assert len(search["budgets"]) == 1, "the second window was never walked" + + +def test_an_element_that_cannot_be_converted_never_matches(backend): + broken = RawElement(cached=True) + del broken.CachedName + assert backend._raw_matches(broken, {"name": "OK"}, cached=True) is False + + +# --- converting an element ---------------------------------------------------- + +def test_an_element_carries_its_name_role_and_geometry(): + raw = _element(name="OK", control_type=50000, + rect=Rect(10, 20, 110, 70), process_id=7, + automation_id="okButton") + element = _convert_uia(raw) + assert element.name == "OK" + assert element.role == "ControlType_50000" + assert element.bounds == (10, 20, 100, 50) + assert element.native_id == "okButton" + assert element.process_id == 7 + + +def test_a_role_is_reported_as_the_raw_control_type(): + # Translation to a friendly name is a separate step on purpose; the + # element carries what UIA said. + assert _convert_uia(_element(control_type=50004)).role == "ControlType_50004" + + +def test_the_owning_application_is_named_from_its_pid(): + assert _convert_uia(_element(process_id=7)).app_name == "app7.exe" + + +def test_a_rectangle_that_is_inside_out_reads_as_no_size(): + # An offscreen or collapsed control reports right < left; a negative + # width would put the element's centre outside the screen. + element = _convert_uia(_element(rect=Rect(100, 100, 10, 10))) + assert element.bounds == (100, 100, 0, 0) + + +def test_a_disabled_control_is_carried_as_disabled(): + assert _convert_uia(_element(enabled=False)).enabled is False + + +def test_a_cached_element_is_read_from_its_cached_properties(): + # The whole cost of a desktop listing is `Current*` reads, one + # cross-process call each; cached elements carry them already. + raw = _element(name="OK", cached=True) + assert _convert_uia(raw, cached=True).name == "OK" + assert _convert_uia(raw, cached=False) is None, "no Current* on it" + + +def test_an_element_that_went_away_mid_conversion_is_none(): + raw = _element(name="OK") + del raw.CurrentName + assert _convert_uia(raw) is None + + +def test_an_element_with_no_name_converts_with_an_empty_one(): + raw = RawElement(name=None, control_type=None, rect=Rect(), + process_id=None, automation_id=None, enabled=False) + element = _convert_uia(raw) + assert (element.name, element.role) == ("", "ControlType_0") + + +def test_a_name_read_that_fails_reads_as_empty(): + raw = _element(name="OK") + del raw.CurrentName + assert _safe_name(raw) == "" + + +def test_a_name_is_read_from_the_current_property(): + assert _safe_name(_element(name="OK")) == "OK" + + +# --- the rich property read --------------------------------------------------- + +def test_the_rich_properties_are_read_one_by_one(): + raw = types.SimpleNamespace( + CurrentIsEnabled=True, CurrentIsOffscreen=False, + CurrentHelpText="press me", CurrentItemStatus="busy", + CurrentAcceleratorKey="Ctrl+S", CurrentAccessKey="s", + CurrentOrientation=1, + ) + assert _read_properties(raw) == { + "enabled": True, "offscreen": False, "help_text": "press me", + "item_status": "busy", "accelerator_key": "Ctrl+S", + "access_key": "s", "orientation": 1, + } + + +def test_a_property_the_provider_will_not_answer_reads_as_none(): + # Each key is present either way: a missing key and a null value are + # different answers to the caller. + assert _read_properties(types.SimpleNamespace()) == { + "enabled": None, "offscreen": None, "help_text": None, + "item_status": None, "accelerator_key": None, "access_key": None, + "orientation": None, + } + + +def test_a_property_of_the_wrong_type_reads_as_none(): + raw = types.SimpleNamespace(CurrentOrientation="sideways") + assert _read_properties(raw)["orientation"] is None + + +def test_get_properties_needs_a_control_to_read_them_from(backend, listing): + assert backend.get_properties(name="OK") is None + + +def test_get_properties_reads_the_control_it_found(backend, listing): + window = _element(name="Editor") + window.CurrentHelpText = "the editor" + listing["roots"] = [window] + assert backend.get_properties(name="Editor")["help_text"] == "the editor" + + +# --- naming a process --------------------------------------------------------- + +@pytest.mark.parametrize("pid", [0, -1]) +def test_a_process_id_that_is_not_one_has_no_name(pid): + assert _process_name(pid) == "" + + +@pytest.mark.skipif(sys.platform == "win32", + reason="on Windows the real lookup runs") +def test_off_windows_no_process_lookup_is_attempted(): + assert _process_name(4321) == "" + + +@pytest.mark.skipif(sys.platform != "win32", reason="Win32 API") +def test_this_interpreters_own_process_names_itself(): + import os + assert _process_name(os.getpid()).lower().startswith("python") + + +@pytest.mark.skipif(sys.platform != "win32", reason="Win32 API") +def test_a_process_that_is_not_there_has_no_name(): + # OpenProcess fails and the name is empty rather than an exception out of + # the middle of a desktop listing. + assert _process_name(0x7FFFFFFF) == "" + + +def test_the_process_name_lookup_is_cached(): + # A desktop listing asks about the same handful of pids thousands of + # times, and each miss is three Win32 round trips. + assert hasattr(_process_name, "cache_info") + before = _process_name.cache_info() + _process_name(0) + _process_name(0) + after = _process_name.cache_info() + assert after.hits > before.hits diff --git a/test/unit_test/headless/test_adapter_registry_sweep.py b/test/unit_test/headless/test_adapter_registry_sweep.py new file mode 100644 index 00000000..52bc39cf --- /dev/null +++ b/test/unit_test/headless/test_adapter_registry_sweep.py @@ -0,0 +1,426 @@ +"""The two adapter registries, swept for what only a real call would reveal. + +``utils/mcp_server/tools/_handlers.py`` and the ``AC_*`` dispatch table in +``utils/executor/action_executor.py`` are the same layer twice: about a thousand +short functions whose whole job is to take a client's arguments, call one +headless function, and hand back something that survives ``json.dumps``. Each is +two to eight lines, so the per-feature tests that touch them are testing the +feature; the adapter itself -- the wiring -- is checked by nobody, and it fails +only when a client calls it. + +The technique both sweeps share is what makes them possible at all: **the callee +is replaced by a stub built from its own return annotation**. Before the typing +contract was emptied of exemptions on 2026-08-22 that could not be done -- 136 +modules had nothing to read. Now "what is this adapter entitled to assume?" is a +question a program can answer, so an adapter can be run against exactly what its +callee promises, no more and no less, with no mouse, no display and no network +in the picture. An adapter that needs *more* than the promise shows up here +rather than in front of a client. + +Arguments never come from the adapter's source. The MCP sweep takes them from +the tool's declared JSON schema in ``_factories.py``; the executor sweep takes +them from the Script Builder's ``command_schema.py`` where a command has an +entry, and from the adapter's own parameter annotations where it does not. Both +sources are maintained in different files from the adapters, so neither sweep +can pass by restating the code it checks. + +What this catches: a required property with no matching parameter, a declared +property the adapter rejects, a mandatory parameter the schema never fills, a +swapped or dropped argument, a "pass-through" that is not one, a command name +the table does not resolve, and a return value the JSON boundary cannot encode. + +Out of scope by design: an adapter that reaches into two project modules +(it composes them rather than normalising one), and one that imports a +third-party module (it picks its own backend, so what it returns depends on the +machine -- the opposite of what a sweep can assert). +""" +import inspect +import json +import threading +import typing +from typing import Any, Callable, Dict, List + +import pytest + +from headless._contract_sweep import ( + NONE_TYPE, contract_stubs, delegation, install_recorder, install_stubs, + is_serialisable, sample_value, +) + +from je_auto_control.gui.script_builder.command_schema import ( + FieldSpec, FieldType, _build_specs, +) +from je_auto_control.utils.executor.action_executor import executor +from je_auto_control.utils.mcp_server.tools import ( + MCPContent, MCPTool, build_default_tool_registry, +) + + +# MCP adapters that need more than their callee's return annotation promises. +# Each is a real coupling the type contract does not express, not a stub defect: +_NEEDS_MORE_THAN_THE_CONTRACT = { + # Uses the return value as a context manager; the annotation is untyped. + "ac_list_monitors", + # Indexes a key out of a Dict[str, Any] the annotation cannot promise. + "ac_tween_drag", + "ac_voice_dispatch", + # Its anchor's `kind` selects a locator backend, and the first value the + # enum offers is `image` -- so the call leaves the adapter and goes into + # OpenCV template matching against a real screen. + "ac_anchor_click", + # Needs a viewer that registered earlier. The registry it is handed is + # real and empty, and refusing an unknown viewer is what it is for. + "ac_presence_update_cursor", + "ac_presence_set_role", + # Needs an interaction recorded into the cassette first; a fresh one + # misses by design. + "ac_http_replay", +} + + +@pytest.fixture(autouse=True) +def _in_a_directory_of_its_own(tmp_path, monkeypatch): + """Run every sweep case in an empty directory. + + An adapter whose callee is a class builds the real object out of the + client's own arguments, and some of those objects are stores: a + checkpoint store handed the sample file path creates a SQLite database + where it stands. In the repository that leaves files behind and makes one + case depend on whether another ran first; here each case gets a directory + nobody else can see. + """ + monkeypatch.chdir(tmp_path) + + +@pytest.fixture(autouse=True) +def _leaves_no_background_work_behind(): + """No sweep case may outlive itself in a thread. + + Running the genuine object rather than a stand-in means an adapter whose + job is to *start* something really starts it: `AC_usb_watch_start` leaves + a hotplug poller running, and on Windows that poller shells out to + PowerShell every interval. Any later test that patches `subprocess.run` + process-wide and reads the first call it recorded then sees the poller's + argv instead of its own -- which is how `test_wayland_libei` failed on one + square of the matrix and nowhere else. + + The sweep stops what it starts, below; this is the guard that says so, and + it names the next adapter to grow a thread instead of letting it become + somebody else's flake. + """ + before = {thread.ident for thread in threading.enumerate()} + yield + left = [thread.name for thread in threading.enumerate() + if thread.ident not in before and thread.is_alive()] + assert not left, f"the case left these threads running: {left}" + + +def _stop_whatever_it_started(name: str, dispatch) -> None: + """Run the `_stop` sibling of a `_start` adapter, where there is one. + + Every `_start` in either registry has one, which is what makes this a + convention rather than a special case -- and running it is coverage of + the stop adapter too, from the only state where stopping means anything. + """ + if not name.endswith("_start"): + return + dispatch(name[:-len("_start")] + "_stop") + + +# === Reading a value out of a declared type ================================= + +# Sample values by JSON-Schema type. Strings carry their property name so a +# swapped pair of same-typed arguments shows as a mismatch, not as two equal +# placeholders. +# One sample per Script Builder field type; a field's own default wins where it +# has one, so enums and paths stay inside the values the editor would offer. +_FIELD_SAMPLES: Dict[FieldType, Callable[[FieldSpec], Any]] = { + FieldType.STRING: lambda field: f"value-for-{field.name}", + FieldType.INT: lambda field: 3, + FieldType.FLOAT: lambda field: 1.5, + FieldType.BOOL: lambda field: True, + FieldType.ENUM: lambda field: field.choices[0] if field.choices else "", + FieldType.FILE_PATH: lambda field: "sample.txt", + FieldType.RGB: lambda field: [1, 2, 3], +} + + +def _field_value(field: FieldSpec) -> Any: + """Return a value the visual editor could have produced for one field.""" + if field.default is not None and field.field_type is not FieldType.RGB: + return field.default + return _FIELD_SAMPLES[field.field_type](field) + + +def _annotated_value(annotation: Any, name: str) -> Any: + """Return an argument value for a parameter with no schema field. + + Structured parameters (the editor leaves those to its raw JSON view) get an + empty container; scalars get a sample of their annotated type. ``Any`` and + unannotated parameters get None, which is why a command carrying one is only + swept when the schema names it. + """ + origin = typing.get_origin(annotation) + arguments = typing.get_args(annotation) + if origin is typing.Union: + if NONE_TYPE in arguments: + return None + return _annotated_value(arguments[0], name) + if origin in (list, tuple, set): + return [] + if origin is dict: + return {} + scalars = {int: 3, float: 1.5, bool: True, str: f"value-for-{name}"} + return scalars.get(annotation) + + +# === The MCP tool registry ================================================== + +def _tool_arguments(schema: Dict[str, Any], + required_only: bool) -> Dict[str, Any]: + """Build a call payload from a tool's input schema.""" + properties = schema.get("properties") or {} + names = (schema.get("required") or []) if required_only else list(properties) + return {name: sample_value(properties[name], name) + for name in names if name in properties} + + +def _unique_by_handler(tools: List[MCPTool]) -> List[MCPTool]: + """Drop the short aliases, which are copies pointing at the same handler.""" + seen = set() + unique = [] + for tool in tools: + if tool.handler in seen: + continue + seen.add(tool.handler) + unique.append(tool) + return unique + + +def _invoke_by_name(name: str) -> None: + """Call one tool by name with what its own schema declares, if it exists.""" + for tool in REGISTRY: + if tool.name == name: + tool.invoke(_tool_arguments(tool.input_schema, required_only=False)) + return + + +REGISTRY = build_default_tool_registry(read_only=False, aliases=False) +_TOOLS = _unique_by_handler(REGISTRY) +DELEGATING = [(tool, *delegation(tool.handler)) for tool in _TOOLS + if delegation(tool.handler) is not None] +STUBBABLE = [tool for tool in _TOOLS + if contract_stubs(tool.handler) is not None] + + +@pytest.mark.parametrize("case", DELEGATING, ids=lambda case: case[0].name) +def test_required_arguments_alone_make_the_tool_callable(case, monkeypatch): + """A client sending exactly the schema's required properties succeeds.""" + tool, module_path, attribute = case + record, sentinel, _original = install_recorder( + monkeypatch, module_path, attribute) + payload = _tool_arguments(tool.input_schema, required_only=True) + assert tool.invoke(payload) is sentinel + assert record, f"{tool.name} never reached {module_path}.{attribute}" + + +@pytest.mark.parametrize("case", DELEGATING, ids=lambda case: case[0].name) +def test_declared_arguments_reach_the_delegate_unchanged(case, monkeypatch): + """Every property the schema declares is accepted and forwarded intact.""" + tool, module_path, attribute = case + record, sentinel, original = install_recorder( + monkeypatch, module_path, attribute) + payload = _tool_arguments(tool.input_schema, required_only=False) + assert tool.invoke(payload) is sentinel + try: + bound = inspect.signature(original).bind( + *record["args"], **record["kwargs"]).arguments + except TypeError as error: # the real delegate would have rejected it + pytest.fail(f"{tool.name} calls {attribute} wrongly: {error}") + for name, value in payload.items(): + if name in bound: + assert bound[name] == value, ( + f"{tool.name} sent {value!r} but {attribute} bound " + f"{bound[name]!r} to {name}") + else: + # The delegate renames the parameter; the value must still arrive. + assert any(seen == value for seen in bound.values()), ( + f"{tool.name} dropped {name}={value!r} on the way to " + f"{module_path}.{attribute}") + + +@pytest.mark.parametrize("tool", STUBBABLE, ids=lambda tool: tool.name) +def test_tool_result_survives_json(tool, monkeypatch): + """An adapter fed exactly what its callee promises returns JSON.""" + if tool.name in _NEEDS_MORE_THAN_THE_CONTRACT: + pytest.skip("documented: needs more than the callee's annotation") + install_stubs(monkeypatch, contract_stubs(tool.handler)) + result = tool.invoke(_tool_arguments(tool.input_schema, + required_only=False)) + _stop_whatever_it_started(tool.name, lambda stop: _invoke_by_name(stop)) + assert is_serialisable(result, (MCPContent,)), ( + f"{tool.name} returned {type(result).__name__}, which json.dumps " + "cannot encode") + + +def test_the_documented_exceptions_are_still_needed(monkeypatch): + """A named exception that starts passing is stale and must be removed.""" + still_failing = set() + for tool in STUBBABLE: + if tool.name not in _NEEDS_MORE_THAN_THE_CONTRACT: + continue + stubs = contract_stubs(tool.handler) + with monkeypatch.context() as patch: + install_stubs(patch, stubs) + try: + result = tool.invoke( + _tool_arguments(tool.input_schema, required_only=False)) + except Exception: # noqa: BLE001 # reason: any failure keeps it + still_failing.add(tool.name) + else: + if not is_serialisable(result, (MCPContent,)): + still_failing.add(tool.name) + assert still_failing == _NEEDS_MORE_THAN_THE_CONTRACT, ( + "these entries now pass and should be deleted: " + f"{sorted(_NEEDS_MORE_THAN_THE_CONTRACT - still_failing)}") + + +# === The AC_* dispatch table ================================================ + +SPECS = _build_specs() +SPECS_BY_COMMAND = {spec.command: spec for spec in SPECS} + + +def _mandatory_parameters(adapter: Any) -> Dict[str, Any]: + """Return the parameters a caller must supply, mapped to their annotation.""" + parameters = inspect.signature(adapter).parameters + return {name: parameter.annotation + for name, parameter in parameters.items() + if parameter.default is parameter.empty + and parameter.kind in (parameter.POSITIONAL_OR_KEYWORD, + parameter.KEYWORD_ONLY)} + + +def _command_arguments(command: str, adapter: Any) -> Dict[str, Any]: + """Build the action payload a client would send for one command.""" + spec = SPECS_BY_COMMAND.get(command) + payload = ({field.name: _field_value(field) for field in spec.fields} + if spec is not None else {}) + for name, annotation in _mandatory_parameters(adapter).items(): + if name not in payload: + payload[name] = _annotated_value(annotation, name) + return payload + + +def _is_drivable(command: str, adapter: Any) -> bool: + """Return True when the sweep can supply every argument a command needs. + + Without a schema entry, a mandatory parameter annotated ``Any`` (or not + annotated at all) leaves nothing to build a value from, and passing None + would test the adapter's None-handling rather than its wiring. + """ + if contract_stubs(adapter) is None: + return False + if any(parameter.kind is parameter.VAR_KEYWORD + for parameter in inspect.signature(adapter).parameters.values()): + return False + if command in SPECS_BY_COMMAND: + return True + return all(annotation not in (inspect.Parameter.empty, typing.Any) + for annotation in _mandatory_parameters(adapter).values()) + + +SWEEPABLE = sorted(command for command, adapter in executor.event_dict.items() + if _is_drivable(command, adapter)) + + +@pytest.mark.parametrize("command", SWEEPABLE) +def test_command_dispatches_and_records_json(command, monkeypatch): + """The command runs from a client's arguments and lands JSON in the record.""" + adapter = executor.event_dict[command] + install_stubs(monkeypatch, contract_stubs(adapter)) + payload = _command_arguments(command, adapter) + record = executor.execute_action([[command, payload]]) + _stop_whatever_it_started(command, lambda stop: executor.execute_action( + [[stop, _command_arguments(stop, executor.event_dict[stop])]])) + assert record, f"{command} produced no execution record" + try: + json.dumps(record) + except (TypeError, ValueError) as error: + pytest.fail(f"{command} put an unencodable value in the record: " + f"{error}") + + +def test_every_schema_field_names_a_parameter_of_its_command(): + """The editor cannot offer a field the executor would reject.""" + unknown = [] + for spec in SPECS: + adapter = executor.event_dict.get(spec.command) + if adapter is None: + continue + parameters = inspect.signature(adapter).parameters + if any(parameter.kind is parameter.VAR_KEYWORD + for parameter in parameters.values()): + continue + unknown.extend(f"{spec.command}.{field.name}" for field in spec.fields + if field.name not in parameters) + assert not unknown, f"fields no adapter accepts: {unknown}" + + +def test_every_schema_command_is_in_the_dispatch_table(): + """A command the editor can emit must be a command the executor knows.""" + known = executor.known_commands() + missing = [spec.command for spec in SPECS if spec.command not in known] + assert not missing, f"schema commands the executor cannot run: {missing}" + + +def _unfielded_mandatory(spec: Any, adapter: Any) -> List[str]: + """Return this command's mandatory scalar parameters that have no field.""" + parameters = inspect.signature(adapter).parameters + if any(parameter.kind is parameter.VAR_KEYWORD + for parameter in parameters.values()): + return [] + fielded = {field.name for field in spec.fields} + gaps = [] + for name, annotation in _mandatory_parameters(adapter).items(): + if name in fielded: + continue + structured = (typing.get_origin(annotation) is not None + or annotation is typing.Any) + if not structured: + gaps.append(f"{spec.command}.{name}: {annotation}") + return gaps + + +def test_unfielded_mandatory_parameters_are_all_structured(): + """A mandatory parameter with no field must be one no field could express. + + The editor's field types are all scalars, so list- and dict-shaped + parameters are filled from its raw JSON view instead. A mandatory *scalar* + with no field is different: it means the editor emits an action that raises + ``TypeError`` the first time it runs. + """ + scalar_gaps = [] + for spec in SPECS: + adapter = executor.event_dict.get(spec.command) + if adapter is not None: + scalar_gaps.extend(_unfielded_mandatory(spec, adapter)) + assert not scalar_gaps, ( + f"mandatory scalar parameters the editor never fills: {scalar_gaps}") + + +# === The sweeps have to keep finding things ================================= + +def test_both_registries_are_swept_in_bulk(): + """Guard the discovery itself: a broken matcher would sweep nothing.""" + tools = len(REGISTRY) + assert len(DELEGATING) > tools // 3, ( + f"only {len(DELEGATING)} of {tools} tools matched the pure-delegation " + "shape -- the AST matcher has probably drifted") + assert len(STUBBABLE) > tools // 2, ( + f"only {len(STUBBABLE)} of {tools} tools could be stubbed from their " + "callee's annotations -- the typing contract or the matcher has drifted") + commands = len(executor.event_dict) + assert len(SWEEPABLE) > commands // 5, ( + f"only {len(SWEEPABLE)} of {commands} commands matched the " + "single-project-import shape -- the matcher has probably drifted") diff --git a/test/unit_test/headless/test_hotkey_backend_linux.py b/test/unit_test/headless/test_hotkey_backend_linux.py new file mode 100644 index 00000000..1e05f243 --- /dev/null +++ b/test/unit_test/headless/test_hotkey_backend_linux.py @@ -0,0 +1,367 @@ +"""Grabbing a hotkey on X11, from any square rather than only Linux. + +`utils/hotkey/backends/` is the last of the project's backend seams with a +real hole in it -- accessibility and window management are covered now, and +the OCR / vision / LLM / agent seams were already close to full. This is the +X11 half: 138 statements at 24%. + +The same fact makes it reachable: every `Xlib` import is inside a function, +so a stub in `sys.modules` (`_xlib_stub.py`) drives the whole thing from all +nine CI squares. + +What the grab has to get right, and what breaks silently if it does not: + +* **A hotkey is grabbed four times, not once.** X reports NumLock and + CapsLock as modifier bits in the event state, and a grab registered + without them simply does not match while either is on. So every combo is + registered under all four lock combinations, and the match ignores those + same bits again on the way back. +* **A failed grab has to roll back the ones that already took.** X refuses a + grab another client already holds, and it refuses it per lock-variant -- + so a combo can be half-grabbed. Leaving those held, with `_registered` + never updated, leaks the grab and spams `BadAccess` on every following + poll. +* **The match is exact on the modifiers, not a superset.** `ctrl+k` must not + fire on `ctrl+shift+k`: the state has to equal the mask once the lock bits + are masked out, or every binding shadows the ones above it. + +The stub's constants and keysyms carry their real values, and +`test_xlib_stub_values.py` compares every one of them against the installed +library on the Linux squares. +""" +from __future__ import annotations + +import threading + +import pytest + +from headless import _xlib_stub +from je_auto_control.utils.hotkey.backends.linux_backend import ( + LinuxHotkeyBackend, _combo_to_x11, _lock_all_mask, _lock_mask_variants, +) +from je_auto_control.utils.hotkey.hotkey_daemon import ( + BackendContext, HotkeyBinding, +) + +_X = _xlib_stub.X_CONSTANTS +CTRL = _X["ControlMask"] +SHIFT = _X["ShiftMask"] +ALT = _X["Mod1Mask"] +SUPER = _X["Mod4Mask"] +NUM_LOCK = _X["Mod2Mask"] +CAPS_LOCK = _X["LockMask"] + +#: What `_combo_to_x11` resolves "k" to, given the keycodes installed below. +K_KEYCODE = 45 + + +@pytest.fixture +def display(monkeypatch): + """An X display that resolves the keysyms these combos use.""" + fake = _xlib_stub.install(monkeypatch) + fake.keycodes = { + _xlib_stub.KEYSYMS["k"]: K_KEYCODE, + _xlib_stub.KEYSYMS["q"]: 24, + _xlib_stub.KEYSYMS["Return"]: 36, + _xlib_stub.KEYSYMS["F5"]: 71, + _xlib_stub.KEYSYMS["Page_Up"]: 112, + } + return fake + + +@pytest.fixture +def backend(): + return LinuxHotkeyBackend() + + +def _binding(binding_id="b1", combo="ctrl+alt+k"): + return HotkeyBinding(binding_id=binding_id, combo=combo, + script_path="script.json") + + +def _context(bindings, fired=None, stop_after=1): + """A context whose stop event trips after `stop_after` polls.""" + stop = threading.Event() + state = {"polls": 0} + + def _get_bindings(): + state["polls"] += 1 + if state["polls"] >= stop_after: + stop.set() + return list(bindings) + + return BackendContext(stop_event=stop, get_bindings=_get_bindings, + fire=(fired if fired is not None else []).append) + + +# --- turning a combo into a grab ---------------------------------------------- + +@pytest.mark.parametrize("combo,mask", [ + ("k", 0), + ("ctrl+k", CTRL), + ("shift+k", SHIFT), + ("alt+k", ALT), + ("win+k", SUPER), + ("ctrl+alt+k", CTRL | ALT), + ("ctrl+shift+alt+win+k", CTRL | SHIFT | ALT | SUPER), +]) +def test_a_combo_becomes_an_x11_modifier_mask(display, combo, mask): + assert _combo_to_x11(combo) == (mask, K_KEYCODE) + + +@pytest.mark.parametrize("combo,keycode", [ + ("ctrl+enter", 36), ("ctrl+return", 36), ("ctrl+f5", 71), + ("ctrl+pageup", 112), +]) +def test_a_named_key_resolves_through_its_keysym(display, combo, keycode): + # "pageup" is not what X calls it -- `Page_Up` is -- so the table in the + # module is the only thing between the user's spelling and a grab. + assert _combo_to_x11(combo)[1] == keycode + + +def test_a_single_character_key_needs_no_table_entry(display): + assert _combo_to_x11("ctrl+Q")[1] == 24, "upper case is lowered first" + + +def test_a_multi_character_key_x_does_not_know_is_refused(display): + with pytest.raises(ValueError, match="unsupported hotkey key"): + _combo_to_x11("ctrl+mediaplay") + + +def test_a_key_with_no_keysym_is_refused(display): + # `string_to_keysym` answers 0 for a name X has never heard of; grabbing + # keycode 0 would register a hotkey on nothing. + with pytest.raises(ValueError, match="unknown X keysym"): + _combo_to_x11("ctrl+¥") + + +def test_resolving_a_combo_closes_the_display_it_opened(display): + _combo_to_x11("ctrl+k") + assert display.closed is True + + +# --- the lock-mask variants --------------------------------------------------- + +def test_a_hotkey_is_grabbed_under_every_lock_combination(display): + # X reports NumLock and CapsLock as modifier bits, and a grab registered + # without them does not match while either is on. + assert _lock_mask_variants() == [0, NUM_LOCK, CAPS_LOCK, + NUM_LOCK | CAPS_LOCK] + + +def test_the_lock_bits_are_masked_out_of_the_event_state(display): + assert _lock_all_mask() == NUM_LOCK | CAPS_LOCK + + +def test_without_xlib_the_variants_degrade_to_the_plain_grab(monkeypatch): + import sys + monkeypatch.setitem(sys.modules, "Xlib", None) + assert _lock_mask_variants() == [0] + assert _lock_all_mask() == 0 + + +# --- registering and unregistering -------------------------------------------- + +def test_a_new_binding_is_grabbed_four_times(backend, display): + backend._sync(display, display.root, [_binding()]) + assert display.root.grabbed == [ + (K_KEYCODE, CTRL | ALT | extra) for extra in _lock_mask_variants() + ] + + +def test_a_grab_owns_the_key_rather_than_observing_it(backend, display): + # `owner_events=True` with async modes is what consumes the key, matching + # Windows `RegisterHotKey` semantics. + backend._sync(display, display.root, [_binding()]) + _keycode, _mask, owner_events, pointer, keyboard = display.root.grabs[0] + assert owner_events is True + assert pointer == keyboard == _X["GrabModeAsync"] + + +def test_a_binding_that_has_not_changed_is_not_re_grabbed(backend, display): + bindings = [_binding()] + backend._sync(display, display.root, bindings) + before = len(display.root.grabs) + backend._sync(display, display.root, bindings) + assert len(display.root.grabs) == before + + +def test_a_binding_whose_combo_changed_releases_the_old_key(backend, + display): + # Until 2026-08-24 the old grab was only forgotten, never released: + # the previous combo stayed consumed from every application, fired + # nothing, and `_ungrab_all` could not free what it no longer knew + # about. The Windows backend has always unregistered here. + backend._sync(display, display.root, [_binding(combo="ctrl+k")]) + backend._sync(display, display.root, [_binding(combo="ctrl+q")]) + assert (24, CTRL) in display.root.grabbed + assert (K_KEYCODE, CTRL) not in display.root.grabbed + assert backend._registered["b1"][0] == "ctrl+q" + + +def test_a_binding_that_disappeared_is_ungrabbed(backend, display): + backend._sync(display, display.root, [_binding()]) + backend._sync(display, display.root, []) + assert display.root.grabbed == [] + assert backend._registered == {} + + +def test_an_unparseable_combo_is_logged_and_skipped(backend, display): + backend._sync(display, display.root, + [_binding(combo="ctrl+mediaplay"), _binding("b2", "ctrl+k")]) + assert list(backend._registered) == ["b2"], "the good one still took" + + +def test_a_refused_grab_rolls_back_the_variants_that_took(backend, display): + # X refuses per lock-variant, so a combo can be half-grabbed. Leaving + # those held with `_registered` never updated leaks the grab and spams + # BadAccess on every following poll. + display.root.grab_errors[(K_KEYCODE, CTRL | ALT | CAPS_LOCK)] = ( + RuntimeError("BadAccess")) + backend._sync(display, display.root, [_binding()]) + assert display.root.grabbed == [] + assert backend._registered == {} + + +def test_a_rollback_that_itself_fails_does_not_escape(backend, display): + display.root.grab_errors[(K_KEYCODE, CTRL | ALT | NUM_LOCK)] = ( + RuntimeError("BadAccess")) + display.root.ungrab_errors[(K_KEYCODE, CTRL | ALT)] = ( + RuntimeError("BadWindow")) + backend._sync(display, display.root, [_binding()]) + assert backend._registered == {} + + +def test_an_ungrab_that_fails_is_not_fatal(backend, display): + backend._sync(display, display.root, [_binding()]) + display.root.ungrab_errors[(K_KEYCODE, CTRL | ALT)] = ( + RuntimeError("BadWindow")) + backend._sync(display, display.root, []) + assert backend._registered == {} + + +def test_a_sync_flushes_the_grabs_to_the_server(backend, display): + # X buffers requests; grabs that are never synced are grabs the server + # has not made yet. + before = display.syncs + backend._sync(display, display.root, [_binding()]) + assert display.syncs > before + + +# --- matching an event back to a binding -------------------------------------- + +def _key_press(keycode=K_KEYCODE, state=CTRL | ALT): + import types as _types + return _types.SimpleNamespace(type=_X["KeyPress"], detail=keycode, + state=state) + + +def test_a_matching_key_press_fires_its_binding(backend, display): + backend._sync(display, display.root, [_binding()]) + display.events = [_key_press()] + fired = [] + backend._drain(display, fired.append) + assert fired == ["b1"] + + +@pytest.mark.parametrize("extra", [NUM_LOCK, CAPS_LOCK, NUM_LOCK | CAPS_LOCK]) +def test_a_hotkey_fires_with_the_locks_on(backend, display, extra): + backend._sync(display, display.root, [_binding()]) + display.events = [_key_press(state=CTRL | ALT | extra)] + fired = [] + backend._drain(display, fired.append) + assert fired == ["b1"] + + +def test_a_press_with_an_extra_modifier_does_not_fire(backend, display): + # `ctrl+k` must not fire on `ctrl+shift+k`, or every binding shadows the + # ones with more modifiers. + backend._sync(display, display.root, [_binding(combo="ctrl+k")]) + display.events = [_key_press(state=CTRL | SHIFT)] + fired = [] + backend._drain(display, fired.append) + assert fired == [] + + +def test_a_press_with_a_missing_modifier_does_not_fire(backend, display): + backend._sync(display, display.root, [_binding()]) + display.events = [_key_press(state=CTRL)] + fired = [] + backend._drain(display, fired.append) + assert fired == [] + + +def test_a_press_of_another_key_does_not_fire(backend, display): + backend._sync(display, display.root, [_binding()]) + display.events = [_key_press(keycode=99)] + fired = [] + backend._drain(display, fired.append) + assert fired == [] + + +def test_an_event_that_is_not_a_key_press_is_ignored(backend, display): + import types as _types + backend._sync(display, display.root, [_binding()]) + display.events = [_types.SimpleNamespace(type=99, detail=K_KEYCODE, + state=CTRL | ALT)] + fired = [] + backend._drain(display, fired.append) + assert fired == [] + + +def test_every_queued_event_is_drained_in_one_poll(backend, display): + backend._sync(display, display.root, [_binding()]) + display.events = [_key_press(), _key_press(), _key_press()] + fired = [] + backend._drain(display, fired.append) + assert fired == ["b1", "b1", "b1"] + assert display.pending_events() == 0 + + +# --- the run loop ------------------------------------------------------------- + +def test_the_loop_asks_the_root_for_key_presses(backend, display): + backend.run_forever(_context([_binding()])) + assert display.root.attributes == {"event_mask": _X["KeyPressMask"]} + + +def test_the_loop_registers_polls_and_tears_down(backend, display): + fired = [] + backend.run_forever(_context([_binding()], fired)) + assert display.root.grabs, "it grabbed while running" + assert display.root.grabbed == [], "and released everything on the way out" + assert backend._registered == {} + assert display.closed is True + + +def test_the_loop_fires_a_hotkey_pressed_while_it_runs(backend, display): + display.events = [_key_press()] + fired = [] + backend.run_forever(_context([_binding()], fired)) + assert fired == ["b1"] + + +def test_a_session_with_no_display_gives_up_quietly(monkeypatch, backend): + # Wayland, or a daemon started outside a session. There is nothing to + # grab, and taking the daemon's thread down with an exception would take + # every other binding with it. + _xlib_stub.install(monkeypatch, + display_error=RuntimeError("no DISPLAY")) + backend.run_forever(_context([_binding()])) + assert backend._registered == {} + + +def test_a_teardown_ungrab_that_fails_does_not_escape(backend, display): + # X11 ungrab races are non-fatal: the window can already be gone. The + # daemon's thread is on its way out and must not raise there. + backend._sync(display, display.root, [_binding()]) + display.root.ungrab_errors = { + (K_KEYCODE, CTRL | ALT | extra): RuntimeError("BadWindow") + for extra in _lock_mask_variants() + } + backend._ungrab_all(display, display.root) + assert backend._registered == {} + + +def test_the_backend_names_the_protocol_it_uses(backend): + assert backend.name == "linux-x11" diff --git a/test/unit_test/headless/test_hotkey_backend_win_mac.py b/test/unit_test/headless/test_hotkey_backend_win_mac.py new file mode 100644 index 00000000..ee458383 --- /dev/null +++ b/test/unit_test/headless/test_hotkey_backend_win_mac.py @@ -0,0 +1,508 @@ +"""Registering a hotkey on Windows and on macOS, and picking a backend. + +The X11 half is in `test_hotkey_backend_linux.py`. These two do the same job +through completely different machinery -- `RegisterHotKey` plus a message +pump on Windows, a `CGEventTap` plus a run loop on macOS -- and both sat at +around 12% and 41%. + +Neither needed a desktop. The Windows backend takes `user32` as an *argument* +to the three methods that do the work, so a recorder drives them anywhere; +only the prologue that builds it needs `ctypes.wintypes`, and that part is +marked Windows-only. The macOS one imports Quartz and CoreFoundation inside +its loop, so stubs in `sys.modules` reach it. + +What the two share, and what these tests are about: + +* **Sync is a diff, not a re-registration.** A binding whose combo has not + changed must not be unregistered and registered again on every poll -- + that is a window during which the hotkey does not work, four times a + second. +* **A combo the parser rejects must not take the others down.** Bindings come + from a user-edited file; one bad line is normal. +* **The teardown has to release what it took.** An OS-level hotkey survives + the process that registered it on neither platform, but a leaked + registration inside a long-lived daemon means a key that is swallowed and + fires nothing. +* **macOS consumes the event by returning None from the tap** and defers the + actual firing to the polling thread, because the tap callback runs on the + run loop and calling a user script there would block every key on the + system until it finished. +""" +from __future__ import annotations + +import sys +import threading +import types + +import pytest + +from je_auto_control.utils.hotkey import backends as backends_mod +from je_auto_control.utils.hotkey.backends import macos_backend as mac +from je_auto_control.utils.hotkey.backends.macos_backend import ( + MacOSHotkeyBackend, _combo_to_macos, _primary_key_to_keycode, +) +from je_auto_control.utils.hotkey.backends.windows_backend import ( + WindowsHotkeyBackend, +) +from je_auto_control.utils.hotkey.hotkey_daemon import ( + BackendContext, HotkeyBinding, parse_combo, +) + + +def _binding(binding_id="b1", combo="ctrl+alt+k"): + return HotkeyBinding(binding_id=binding_id, combo=combo, + script_path="script.json") + + +def _context(bindings, fired=None, stop_after=1): + stop = threading.Event() + state = {"polls": 0} + + def _get_bindings(): + state["polls"] += 1 + if state["polls"] >= stop_after: + stop.set() + return list(bindings) + + return BackendContext(stop_event=stop, get_bindings=_get_bindings, + fire=(fired if fired is not None else []).append) + + +# --- the selector ------------------------------------------------------------- + +@pytest.fixture +def on_platform(monkeypatch): + def _set(name: str): + monkeypatch.setattr(backends_mod.sys, "platform", name) + return _set + + +@pytest.mark.parametrize("platform,expected", [ + ("win32", "windows"), ("darwin", "macos"), ("linux", "linux-x11"), + ("linux2", "linux-x11"), +]) +def test_each_platform_gets_its_own_backend(on_platform, platform, expected): + on_platform(platform) + assert backends_mod.get_backend().name == expected + + +def test_a_platform_with_no_hotkey_backend_says_so(on_platform): + # Unlike the accessibility and window seams there is no null backend + # here: a daemon that silently registered nothing would look like a + # daemon whose hotkeys never fire. + on_platform("sunos5") + with pytest.raises(NotImplementedError, match="sunos5"): + backends_mod.get_backend() + + +# --- Windows: RegisterHotKey -------------------------------------------------- + +class _User32: + """The three user32 entry points the backend calls, as a recorder.""" + + def __init__(self, register_result=True) -> None: + self.registered = [] + self.unregistered = [] + self.register_result = register_result + self.RegisterHotKey = _Stub(self._register) # noqa: N815 + self.UnregisterHotKey = _Stub(self._unregister) # noqa: N815 + self.PeekMessageW = _Stub(lambda *args: 0) # noqa: N815 + + def _register(self, hwnd, reg_id, modifiers, vk): + self.registered.append((reg_id, modifiers, vk)) + return 1 if self.register_result else 0 + + def _unregister(self, hwnd, reg_id): + self.unregistered.append(reg_id) + return 1 + + @property + def held(self): + """Registration ids still outstanding, in the order they were taken.""" + out = [reg_id for reg_id, _mods, _vk in self.registered] + for reg_id in self.unregistered: + if reg_id in out: + out.remove(reg_id) + return out + + +class _Stub: + """A user32 function pointer: callable, and it accepts argtypes/restype.""" + + def __init__(self, call) -> None: + self._call = call + self.argtypes = None + self.restype = None + + def __call__(self, *args): + return self._call(*args) + + +@pytest.fixture +def user32(): + return _User32() + + +@pytest.fixture +def win_backend(): + return WindowsHotkeyBackend() + + +def test_a_new_binding_is_registered_with_its_parsed_combo(win_backend, + user32): + win_backend._sync(user32, [_binding()]) + [(_reg_id, modifiers, vk)] = user32.registered + assert (modifiers, vk) == parse_combo("ctrl+alt+k") + + +def test_each_registration_gets_its_own_id(win_backend, user32): + win_backend._sync(user32, [_binding("b1", "ctrl+k"), + _binding("b2", "ctrl+q")]) + ids = [reg_id for reg_id, _m, _v in user32.registered] + assert len(set(ids)) == 2 + + +def test_a_binding_that_has_not_changed_is_not_re_registered(win_backend, + user32): + bindings = [_binding()] + win_backend._sync(user32, bindings) + win_backend._sync(user32, bindings) + assert len(user32.registered) == 1 + assert user32.unregistered == [], "the hotkey never stopped working" + + +def test_a_binding_whose_combo_changed_releases_the_old_key(win_backend, + user32): + win_backend._sync(user32, [_binding(combo="ctrl+k")]) + first_id = user32.registered[0][0] + win_backend._sync(user32, [_binding(combo="ctrl+q")]) + assert first_id in user32.unregistered + assert len(user32.held) == 1 + + +def test_a_binding_that_disappeared_is_unregistered(win_backend, user32): + win_backend._sync(user32, [_binding()]) + win_backend._sync(user32, []) + assert user32.held == [] + assert win_backend._registered == {} + + +def test_an_unparseable_combo_is_logged_and_skipped(win_backend, user32): + win_backend._sync(user32, [_binding(combo="ctrl+"), _binding("b2", "ctrl+k")]) + assert list(win_backend._registered) == ["b2"] + + +def test_a_registration_windows_refuses_is_not_remembered(win_backend): + # Another application already owns the combo. Remembering it would mean + # never retrying, and reporting it as bound would be a lie. + refusing = _User32(register_result=False) + win_backend._sync(refusing, [_binding()]) + assert win_backend._registered == {} + + +def test_a_hotkey_message_fires_the_binding_it_belongs_to(win_backend, + user32): + win_backend._sync(user32, [_binding("b1", "ctrl+k"), + _binding("b2", "ctrl+q")]) + second_id = user32.registered[1][0] + fired = [] + win_backend._dispatch(second_id, fired.append) + assert fired == ["b2"] + + +def test_a_hotkey_message_for_an_unknown_id_fires_nothing(win_backend, + user32): + win_backend._sync(user32, [_binding()]) + fired = [] + win_backend._dispatch(9999, fired.append) + assert fired == [] + + +@pytest.mark.skipif(sys.platform != "win32", + reason="ctypes.wintypes is Windows-only") +def test_the_message_pump_registers_and_releases(monkeypatch, win_backend): + import ctypes + recorder = _User32() + monkeypatch.setattr(ctypes, "WinDLL", + lambda name, **kwargs: recorder) + win_backend.run_forever(_context([_binding()])) + assert recorder.registered, "it registered while running" + assert recorder.held == [], "and released on the way out" + assert win_backend._registered == {} + + +@pytest.mark.skipif(sys.platform != "win32", + reason="ctypes.wintypes is Windows-only") +def test_a_hotkey_message_reaches_the_binding_through_the_pump(monkeypatch, + win_backend): + import ctypes + recorder = _User32() + messages = {"left": 1} + + def _peek(msg_pointer, hwnd, low, high, remove): + if not messages["left"]: + return 0 + messages["left"] -= 1 + msg = msg_pointer._obj + msg.message = 0x0312 # WM_HOTKEY + msg.wParam = recorder.registered[0][0] + return 1 + + recorder.PeekMessageW = _Stub(_peek) + monkeypatch.setattr(ctypes, "WinDLL", lambda name, **kwargs: recorder) + fired = [] + win_backend.run_forever(_context([_binding()], fired)) + assert fired == ["b1"] + + +@pytest.mark.skipif(sys.platform != "win32", + reason="ctypes.wintypes is Windows-only") +def test_a_message_that_is_not_a_hotkey_is_pumped_past(monkeypatch, + win_backend): + # The queue carries whatever Windows posts to the thread; only WM_HOTKEY + # means one of ours fired. + import ctypes + recorder = _User32() + messages = {"left": 2} + + def _peek(msg_pointer, hwnd, low, high, remove): + if not messages["left"]: + return 0 + messages["left"] -= 1 + msg = msg_pointer._obj + msg.message = 0x0113 if messages["left"] else 0x0312 # WM_TIMER + msg.wParam = recorder.registered[0][0] + return 1 + + recorder.PeekMessageW = _Stub(_peek) + monkeypatch.setattr(ctypes, "WinDLL", lambda name, **kwargs: recorder) + fired = [] + win_backend.run_forever(_context([_binding()], fired)) + assert fired == ["b1"], "the timer was skipped, the hotkey was not" + + +def test_the_windows_backend_names_the_api_it_uses(win_backend): + assert win_backend.name == "windows" + + +# --- macOS: CGEventTap -------------------------------------------------------- + +_FLAG_SHIFT = 1 << 17 +_FLAG_CONTROL = 1 << 18 +_FLAG_ALT = 1 << 19 +_FLAG_CMD = 1 << 20 + + +@pytest.mark.parametrize("combo,mask", [ + ("k", 0), + ("ctrl+k", _FLAG_CONTROL), + ("shift+k", _FLAG_SHIFT), + ("alt+k", _FLAG_ALT), + ("win+k", _FLAG_CMD), + ("ctrl+alt+k", _FLAG_CONTROL | _FLAG_ALT), +]) +def test_a_combo_becomes_a_quartz_flags_mask(combo, mask): + assert _combo_to_macos(combo)[0] == mask + + +@pytest.mark.parametrize("key,keycode", [ + ("a", 0), ("z", 6), ("m", 46), # letters + ("1", 18), ("6", 22), ("0", 29), # digits, in Carbon's odd order + ("return", 36), ("enter", 36), ("space", 49), ("esc", 53), + ("escape", 53), ("f1", 122), ("f12", 111), ("pageup", 116), + ("A", 0), ("F1", 122), # case does not matter +]) +def test_a_key_name_becomes_a_carbon_virtual_keycode(key, keycode): + assert _primary_key_to_keycode(key) == keycode + + +def test_a_key_carbon_has_no_code_for_is_refused(): + with pytest.raises(ValueError, match="unsupported hotkey key"): + _primary_key_to_keycode("mediaplay") + + +@pytest.fixture +def mac_backend(): + return MacOSHotkeyBackend() + + +def test_a_new_binding_is_remembered_with_its_mask_and_keycode(mac_backend): + mac_backend._sync([_binding()]) + assert mac_backend._registered == { + "b1": ("ctrl+alt+k", _FLAG_CONTROL | _FLAG_ALT, 40), + } + + +def test_a_binding_that_has_not_changed_is_left_alone(mac_backend): + bindings = [_binding()] + mac_backend._sync(bindings) + first = mac_backend._registered["b1"] + mac_backend._sync(bindings) + assert mac_backend._registered["b1"] is first + + +def test_a_binding_whose_combo_changed_is_replaced(mac_backend): + mac_backend._sync([_binding(combo="ctrl+k")]) + mac_backend._sync([_binding(combo="ctrl+q")]) + assert mac_backend._registered["b1"][0] == "ctrl+q" + + +def test_a_binding_that_disappeared_is_forgotten(mac_backend): + mac_backend._sync([_binding()]) + mac_backend._sync([]) + assert mac_backend._registered == {} + + +def test_an_unparseable_combo_is_logged_and_skipped_on_macos(mac_backend): + mac_backend._sync([_binding(combo="ctrl+mediaplay"), + _binding("b2", "ctrl+k")]) + assert list(mac_backend._registered) == ["b2"] + + +def test_a_matching_key_event_names_its_binding(mac_backend): + mac_backend._sync([_binding()]) + assert mac_backend._match(40, _FLAG_CONTROL | _FLAG_ALT) == "b1" + + +def test_an_event_with_extra_modifiers_does_not_match(mac_backend): + mac_backend._sync([_binding(combo="ctrl+k")]) + assert mac_backend._match(40, _FLAG_CONTROL | _FLAG_SHIFT) is None + + +def test_an_event_for_another_key_does_not_match(mac_backend): + mac_backend._sync([_binding()]) + assert mac_backend._match(99, _FLAG_CONTROL | _FLAG_ALT) is None + + +def test_pending_fires_are_drained_in_order(mac_backend): + mac_backend._pending_fires = ["b1", "b2"] + fired = [] + mac_backend._drain_fires(fired.append) + assert fired == ["b1", "b2"] + assert mac_backend._pending_fires == [] + + +# --- the macOS run loop ------------------------------------------------------- + +class _Quartz(types.ModuleType): + """The Quartz surface the tap setup touches.""" + + kCGKeyboardEventKeycode = "keycode" # noqa: N815 # reason: Quartz name + kCGEventKeyDown = 10 # noqa: N815 + kCGHIDEventTap = 0 # noqa: N815 + kCGHeadInsertEventTap = 0 # noqa: N815 + kCGEventTapOptionDefault = 0 # noqa: N815 + + def __init__(self, tap="tap") -> None: + super().__init__("Quartz") + self.tap = tap + self.enabled = [] + self.sources_added = [] + self.sources_removed = [] + self.callback = None + self.tap_mask = None + + def CGEventGetIntegerValueField(self, event, field): # noqa: N802 + return event["keycode"] + + def CGEventGetFlags(self, event): # noqa: N802 # reason: Quartz name + return event["flags"] + + def CGEventTapCreate(self, tap_point, place, options, mask, # noqa: N802 + callback, refcon): + self.callback = callback + self.tap_mask = mask + return self.tap + + def CFMachPortCreateRunLoopSource(self, allocator, tap, order): # noqa: N802 + return "source" + + def CFRunLoopGetCurrent(self): # noqa: N802 # reason: Quartz name + return "run-loop" + + def CFRunLoopAddSource(self, loop, source, mode): # noqa: N802 + self.sources_added.append((loop, source, mode)) + + def CFRunLoopRemoveSource(self, loop, source, mode): # noqa: N802 + self.sources_removed.append((loop, source, mode)) + + def CGEventTapEnable(self, tap, enable): # noqa: N802 + self.enabled.append(bool(enable)) + + +@pytest.fixture +def quartz(monkeypatch): + def _install(tap="tap"): + module = _Quartz(tap) + monkeypatch.setitem(sys.modules, "Quartz", module) + core = types.ModuleType("CoreFoundation") + core.kCFRunLoopDefaultMode = "default-mode" + core.CFRunLoopRunInMode = lambda mode, seconds, once: None + monkeypatch.setitem(sys.modules, "CoreFoundation", core) + return module + return _install + + +def test_the_tap_listens_for_key_down_only(mac_backend, quartz): + module = quartz() + mac_backend.run_forever(_context([_binding()])) + assert module.tap_mask == 1 << module.kCGEventKeyDown + + +def test_the_tap_is_enabled_while_running_and_disabled_after(mac_backend, + quartz): + module = quartz() + mac_backend.run_forever(_context([_binding()])) + assert module.enabled == [True, False] + assert module.sources_added and module.sources_removed + + +def test_a_mac_without_accessibility_permission_gives_up_with_a_reason( + mac_backend, quartz): + # `CGEventTapCreate` returns None rather than failing when the grant is + # missing, so this is the only signal there is. + module = quartz(tap=None) + mac_backend.run_forever(_context([_binding()])) + assert module.enabled == [], "nothing to enable, and nothing to remove" + + +def test_a_mac_without_pyobjc_gives_up_quietly(mac_backend, monkeypatch): + monkeypatch.setitem(sys.modules, "Quartz", None) + mac_backend.run_forever(_context([_binding()])) + assert mac_backend._registered == {} + + +def test_a_matching_key_is_consumed_and_queued(mac_backend, quartz): + # Returning None consumes the event; the fire is deferred to the polling + # thread because the callback runs on the run loop, where calling a user + # script would block every key on the system. + module = quartz() + fired = [] + mac_backend.run_forever(_context([_binding()], fired, stop_after=2)) + event = {"keycode": 40, "flags": _FLAG_CONTROL | _FLAG_ALT} + assert module.callback(None, None, event, None) is None + mac_backend._drain_fires(fired.append) + assert fired == ["b1"] + + +def test_a_key_that_matches_nothing_is_passed_through(mac_backend, quartz): + module = quartz() + mac_backend.run_forever(_context([_binding()])) + event = {"keycode": 99, "flags": 0} + assert module.callback(None, None, event, None) is event + + +def test_an_unrelated_modifier_bit_is_masked_off_before_matching(mac_backend, + quartz): + # macOS sets bits for caps lock, the numeric keypad and more; only the + # four modifier flags take part in the comparison. + module = quartz() + mac_backend.run_forever(_context([_binding()])) + event = {"keycode": 40, + "flags": _FLAG_CONTROL | _FLAG_ALT | (1 << 16) | (1 << 21)} + assert module.callback(None, None, event, None) is None + + +def test_the_macos_backend_names_the_api_it_uses(mac_backend): + assert mac_backend.name == "macos" + assert mac.MacOSHotkeyBackend is MacOSHotkeyBackend diff --git a/test/unit_test/headless/test_pyobjc_stub_names.py b/test/unit_test/headless/test_pyobjc_stub_names.py new file mode 100644 index 00000000..6688c03e --- /dev/null +++ b/test/unit_test/headless/test_pyobjc_stub_names.py @@ -0,0 +1,52 @@ +"""Hold the pyobjc stub to the frameworks it stands in for. + +`_pyobjc_stub.py` lets the macOS backends be tested on all nine CI squares by +shadowing `Quartz`, `AppKit` and `ApplicationServices` in `sys.modules`. The +risk that buys is a stub that answers for a name the real framework does not +have -- a test that passes everywhere and a backend that raises +`AttributeError` on the one platform it exists for. + +pyobjc is a hard dependency on Darwin, so on the macOS squares the genuine +frameworks are installed and every name the stub declares is looked up on +them. Elsewhere there is nothing to compare against and these skip. + +Names, not values. Every constant in the stub is either a key into an info +dictionary the stub itself builds or an opaque token handed back to a +function the stub itself provides, so none of their numbers reaches any +arithmetic in the code under test. What matters is that the backend is +spelling real API. +""" +from __future__ import annotations + +import pytest + +from headless import _pyobjc_stub as objc_stub + +pytest.importorskip("Quartz", reason="pyobjc is a macOS-only dependency") + + +@pytest.mark.parametrize("name", sorted(objc_stub.QUARTZ_NAMES)) +def test_every_quartz_name_the_stub_answers_for_exists(name): + import Quartz + assert hasattr(Quartz, name), f"Quartz.{name}" + + +@pytest.mark.parametrize("name", sorted(objc_stub.APPKIT_NAMES)) +def test_every_appkit_name_the_stub_answers_for_exists(name): + import AppKit + assert hasattr(AppKit, name), f"AppKit.{name}" + + +@pytest.mark.parametrize("name", sorted(objc_stub.AX_NAMES)) +def test_every_accessibility_name_the_stub_answers_for_exists(name): + import ApplicationServices + assert hasattr(ApplicationServices, name), f"ApplicationServices.{name}" + + +def test_the_window_list_options_are_the_flags_the_backend_composes(): + # These two are the only Quartz numbers the backend does arithmetic on: + # it ORs them into the argument of CGWindowListCopyWindowInfo. + import Quartz + assert Quartz.kCGWindowListOptionOnScreenOnly == 1 + assert Quartz.kCGWindowListExcludeDesktopElements == 16 + assert Quartz.kCGNullWindowID == 0 diff --git a/test/unit_test/headless/test_remote_desktop_multi_viewer.py b/test/unit_test/headless/test_remote_desktop_multi_viewer.py new file mode 100644 index 00000000..a75f64af --- /dev/null +++ b/test/unit_test/headless/test_remote_desktop_multi_viewer.py @@ -0,0 +1,522 @@ +"""One capture, N viewers: what the coordinator owns and what it delegates. + +`MultiViewerHost` is deliberately thin -- it runs one `WebRTCDesktopHost` +per viewer and forwards almost everything -- but the four things it does +own are the ones that go wrong quietly: + +* **The screen source is shared and reference-counted by hand.** It is + built on the first session and stopped when the last one goes, so the + capture thread outliving every viewer, or being torn down while one is + still watching, are both bugs the tests below would catch. +* **Every callback is re-wrapped to carry a `session_id`.** The GUI gets + `(session_id, state)` where the single-viewer host gives it `(state)`, + and the pending-viewer wrapper has to look the host up *at fire time* -- + the viewer id it reports does not exist when the wrapper is built. +* **The connection timestamp is minted here, not in the host.** It is + written from the auth wrapper, which is the only moment the coordinator + is told a viewer got through. +* **A broadcast must not be stopped by one bad recipient.** `broadcast_file` + skips viewers that never authenticated and keeps going past a viewer + whose channel has died; the count it returns is what the GUI reports. + +`WebRTCDesktopHost`, `ScreenVideoTrack` and `MediaRelay` are all replaced +here: this module is a coordinator, and every one of those three would drag +in a real PeerConnection or a real screen grab. What is *not* faked is the +session bookkeeping, which is the code under test. +""" +from __future__ import annotations + +import pytest + +from headless._webrtc_doubles import Track +from je_auto_control.utils.remote_desktop import multi_viewer as mv +from je_auto_control.utils.remote_desktop.multi_viewer import MultiViewerHost +from je_auto_control.utils.remote_desktop.permissions import SessionPermissions + + +class _FakeRelay: + def __init__(self) -> None: + self.subscribed = [] + + def subscribe(self, track): + proxy = ("proxy", len(self.subscribed)) + self.subscribed.append(track) + return proxy + + +class _FakeHost: + """A `WebRTCDesktopHost` stand-in that records what it was told.""" + + instances = [] + + def __init__(self, **kwargs) -> None: + self.kwargs = kwargs + self.authenticated = False + self.connection_state = "new" + self.pending_viewer_id = None + self.offers = [] + self.answers = [] + self.pushed = [] + self.stopped = False + self.permissions = None + self.calls = [] + self.raise_on = set() + self._pc = None + _FakeHost.instances.append(self) + + def _maybe_raise(self, name: str) -> None: + if name in self.raise_on: + raise RuntimeError(f"{name} failed") + + def create_offer(self, peer_label="remote viewer"): + self.offers.append(peer_label) + return f"sdp-for-{peer_label}" + + def accept_answer(self, answer_sdp): + self.answers.append(answer_sdp) + + def stop(self): + self._maybe_raise("stop") + self.stopped = True + + def approve_pending_viewer(self): + self.calls.append("approve") + + def reject_pending_viewer(self): + self.calls.append("reject") + + def trust_pending_viewer(self, label=""): + self.calls.append(("trust", label)) + + def set_permissions(self, permissions): + self._maybe_raise("set_permissions") + self.permissions = permissions + + def disable_accept_viewer_video(self): + self._maybe_raise("disable_video") + self.calls.append("disable_video") + + def disable_accept_viewer_audio_opus(self): + self._maybe_raise("disable_audio") + self.calls.append("disable_audio") + + def push_file(self, local_path, remote_name=None): + self._maybe_raise("push_file") + self.pushed.append((local_path, remote_name)) + + +@pytest.fixture(autouse=True) +def fake_webrtc(monkeypatch): + """Replace the three things that would touch a screen or a network.""" + _FakeHost.instances = [] + monkeypatch.setattr(mv, "WebRTCDesktopHost", _FakeHost) + monkeypatch.setattr(mv, "ScreenVideoTrack", Track) + monkeypatch.setattr(mv, "MediaRelay", _FakeRelay) + yield + _FakeHost.instances = [] + + +def _host(**kwargs) -> MultiViewerHost: + kwargs.setdefault("token", "shared-secret") + return MultiViewerHost(**kwargs) + + +# --- construction ------------------------------------------------------------- + +def test_a_host_without_a_token_is_refused(): + with pytest.raises(ValueError, match="non-empty token"): + MultiViewerHost(token="") + + +def test_read_only_shorthand_becomes_a_permission_set(): + assert _host(read_only=True).permissions.allow_input is False + assert _host(read_only=False).permissions.allow_input is True + + +def test_explicit_permissions_win_over_the_shorthand(): + # Both arguments reach the GUI's constructor call; the granular one is + # the newer surface and must not be overridden by a stale bool. + permissions = SessionPermissions.from_read_only(False) + host = _host(read_only=True, permissions=permissions) + assert host.permissions is permissions + + +# --- session lifecycle -------------------------------------------------------- + +def test_first_session_builds_the_capture_and_labels_the_peer(): + host = _host() + session_id, offer = host.create_session_offer() + assert len(session_id) == 16, "secrets.token_hex(8)" + assert offer == f"sdp-for-viewer-{session_id[:6]}" + assert host.session_count() == 1 + assert isinstance(host.screen_track(), Track) + + +def test_every_session_subscribes_to_the_same_capture(): + host = _host() + host.create_session_offer() + host.create_session_offer() + track = host.screen_track() + tracks = [inst.kwargs["external_video_track"] + for inst in _FakeHost.instances] + assert len(set(tracks)) == 2, "each viewer gets its own relay proxy" + assert all(t is not track for t in tracks) + assert host.session_count() == 2 + + +def test_the_capture_track_is_built_from_the_shared_config(): + from je_auto_control.utils.remote_desktop.webrtc_transport import ( + WebRTCConfig, + ) + config = WebRTCConfig(monitor_index=2, fps=15, region=(1, 2, 3, 4), + show_cursor=False) + host = _host(config=config) + host.create_session_offer() + assert host.screen_track().kwargs == { + "monitor_index": 2, "fps": 15, "region": (1, 2, 3, 4), + "show_cursor": False, + } + + +def test_sessions_inherit_the_coordinator_settings(): + trust = object() + host = _host(trust_list=trust, ip_whitelist=["10.0.0.1"]) + host.create_session_offer() + kwargs = _FakeHost.instances[0].kwargs + assert kwargs["token"] == "shared-secret" + assert kwargs["trust_list"] is trust + assert kwargs["ip_whitelist"] == ["10.0.0.1"] + + +def test_answers_go_to_the_session_they_belong_to(): + host = _host() + first, _ = host.create_session_offer() + second, _ = host.create_session_offer() + host.accept_session_answer(second, "answer-sdp") + by_id = dict(zip([first, second], _FakeHost.instances)) + assert by_id[second].answers == ["answer-sdp"] + assert by_id[first].answers == [] + + +def test_an_unknown_session_id_is_a_key_error(): + host = _host() + with pytest.raises(KeyError, match="unknown session_id"): + host.accept_session_answer("deadbeef", "answer") + + +def test_the_capture_survives_until_the_last_viewer_leaves(): + host = _host() + first, _ = host.create_session_offer() + second, _ = host.create_session_offer() + track = host.screen_track() + + host.stop_session(first) + assert not track.stopped, "one viewer is still watching" + assert host.session_count() == 1 + + host.stop_session(second) + assert track.stopped + assert host.screen_track() is None + + +def test_stopping_an_unknown_session_is_a_no_op(): + host = _host() + host.create_session_offer() + track = host.screen_track() + host.stop_session("not-a-session") + assert not track.stopped + assert host.session_count() == 1 + + +def test_a_session_that_fails_to_stop_is_still_forgotten(): + # The PeerConnection may already be dead when the GUI asks to close the + # tab; the session must leave the table anyway, or the capture it holds + # a reference to never gets released. + host = _host() + session_id, _ = host.create_session_offer() + track = host.screen_track() + _FakeHost.instances[0].raise_on.add("stop") + host.stop_session(session_id) + assert host.session_count() == 0 + assert track.stopped + + +def test_stop_all_tears_every_session_and_the_capture_down(): + host = _host() + host.create_session_offer() + host.create_session_offer() + track = host.screen_track() + host.stop_all() + assert all(inst.stopped for inst in _FakeHost.instances) + assert track.stopped + assert host.session_count() == 0 + + +def test_a_session_that_fails_to_stop_does_not_strand_the_others(): + host = _host() + host.create_session_offer() + host.create_session_offer() + _FakeHost.instances[0].raise_on.add("stop") + host.stop_all() + assert _FakeHost.instances[1].stopped + assert host.screen_track() is None, "the capture is still released" + + +def test_a_new_session_after_stop_all_builds_a_fresh_capture(): + host = _host() + host.create_session_offer() + first_track = host.screen_track() + host.stop_all() + host.create_session_offer() + assert host.screen_track() is not first_track + + +# --- per-session controls ----------------------------------------------------- + +@pytest.mark.parametrize("method,recorded", [ + ("approve_pending_viewer", "approve"), + ("reject_pending_viewer", "reject"), +]) +def test_pending_viewer_decisions_reach_only_that_session(method, recorded): + host = _host() + first, _ = host.create_session_offer() + host.create_session_offer() + getattr(host, method)(first) + assert _FakeHost.instances[0].calls == [recorded] + assert _FakeHost.instances[1].calls == [] + + +def test_trusting_a_viewer_carries_the_label_through(): + host = _host() + session_id, _ = host.create_session_offer() + host.trust_pending_viewer(session_id, label="Ops laptop") + assert _FakeHost.instances[0].calls == [("trust", "Ops laptop")] + + +def test_pending_viewer_id_is_read_from_the_live_session(): + host = _host() + session_id, _ = host.create_session_offer() + _FakeHost.instances[0].pending_viewer_id = "viewer-7" + assert host.pending_viewer_id(session_id) == "viewer-7" + + +# --- permissions -------------------------------------------------------------- + +def test_permissions_propagate_to_live_sessions_and_to_the_next_one(): + host = _host() + host.create_session_offer() + permissions = SessionPermissions.from_read_only(True) + host.set_permissions(permissions) + assert _FakeHost.instances[0].permissions is permissions + host.create_session_offer() + assert _FakeHost.instances[1].kwargs["permissions"] is permissions + + +def test_set_read_only_is_the_shorthand_for_the_same_thing(): + host = _host() + host.create_session_offer() + host.set_read_only(True) + assert host.permissions.allow_input is False + assert _FakeHost.instances[0].permissions.allow_input is False + + +def test_one_dead_session_does_not_block_the_permission_broadcast(): + host = _host() + host.create_session_offer() + host.create_session_offer() + _FakeHost.instances[0].raise_on.add("set_permissions") + host.set_read_only(True) + assert _FakeHost.instances[1].permissions is not None + + +@pytest.mark.parametrize("method,recorded", [ + ("disable_accept_viewer_video", "disable_video"), + ("disable_accept_viewer_audio_opus", "disable_audio"), +]) +def test_disabling_an_inbound_slot_hits_every_session(method, recorded): + host = _host() + host.create_session_offer() + host.create_session_offer() + getattr(host, method)() + assert all(inst.calls == [recorded] for inst in _FakeHost.instances) + + +@pytest.mark.parametrize("method,failure", [ + ("disable_accept_viewer_video", "disable_video"), + ("disable_accept_viewer_audio_opus", "disable_audio"), +]) +def test_disabling_a_slot_survives_a_session_that_is_already_gone(method, + failure): + host = _host() + host.create_session_offer() + host.create_session_offer() + _FakeHost.instances[0].raise_on.add(failure) + getattr(host, method)() + assert _FakeHost.instances[1].calls, "the live session still got it" + + +# --- broadcast ---------------------------------------------------------------- + +def test_broadcast_reaches_only_authenticated_viewers(): + host = _host() + host.create_session_offer() + host.create_session_offer() + _FakeHost.instances[0].authenticated = True + assert host.broadcast_file("C:/report.txt", remote_name="r.txt") == 1 + assert _FakeHost.instances[0].pushed == [("C:/report.txt", "r.txt")] + assert _FakeHost.instances[1].pushed == [] + + +def test_broadcast_counts_the_viewers_that_actually_took_the_file(): + host = _host() + host.create_session_offer() + host.create_session_offer() + for inst in _FakeHost.instances: + inst.authenticated = True + _FakeHost.instances[0].raise_on.add("push_file") + assert host.broadcast_file("C:/report.txt") == 1 + + +def test_broadcast_with_no_sessions_sends_nothing(): + assert _host().broadcast_file("C:/report.txt") == 0 + + +# --- introspection ------------------------------------------------------------ + +def test_list_sessions_reports_live_state_per_viewer(): + host = _host() + session_id, _ = host.create_session_offer() + inst = _FakeHost.instances[0] + inst.authenticated = True + inst.connection_state = "connected" + inst.pending_viewer_id = "viewer-1" + [row] = host.list_sessions() + assert row["session_id"] == session_id + assert row["authenticated"] is True + assert row["state"] == "connected" + assert row["pending_viewer_id"] == "viewer-1" + assert row["connected_at"] is None, "nobody has authenticated yet" + + +def test_screen_track_is_none_before_any_viewer_arrives(): + assert _host().screen_track() is None + + +def test_first_session_pc_skips_sessions_that_have_no_connection_yet(): + host = _host() + host.create_session_offer() + host.create_session_offer() + pc = object() + _FakeHost.instances[1]._pc = pc + assert host.first_session_pc() is pc + + +def test_first_session_pc_is_none_when_nothing_is_connected(): + host = _host() + host.create_session_offer() + assert host.first_session_pc() is None + + +def test_session_pc_addresses_one_named_session(): + host = _host() + first, _ = host.create_session_offer() + second, _ = host.create_session_offer() + pc = object() + _FakeHost.instances[1]._pc = pc + assert host.session_pc(second) is pc + assert host.session_pc(first) is None + + +def test_session_pc_of_a_closed_session_is_none_rather_than_an_error(): + # The GUI polls this on a timer; a viewer that left between two ticks + # must not raise out of the timer slot. + assert _host().session_pc("gone") is None + + +# --- callback wrappers -------------------------------------------------------- + +def _fire(host_instance, key): + """Invoke the wrapper the coordinator handed to one fake session.""" + host_instance.kwargs[key]() + + +def test_state_changes_are_reported_with_their_session_id(): + seen = [] + host = _host(on_session_state=lambda sid, state: seen.append((sid, state))) + session_id, _ = host.create_session_offer() + _FakeHost.instances[0].kwargs["on_state_change"]("connected") + assert seen == [(session_id, "connected")] + + +def test_no_state_callback_means_no_wrapper_is_installed(): + # The single-viewer host checks this for None before calling it, so + # handing it a wrapper that calls nothing would only cost work. + host = _host() + host.create_session_offer() + assert _FakeHost.instances[0].kwargs["on_state_change"] is None + + +def test_authentication_stamps_the_connection_time_and_notifies(): + seen = [] + host = _host(on_session_authenticated=seen.append) + session_id, _ = host.create_session_offer() + _fire(_FakeHost.instances[0], "on_authenticated") + assert seen == [session_id] + [row] = host.list_sessions() + assert row["connected_at"], "an ISO timestamp minted at auth time" + + +def test_the_connection_time_is_stamped_even_without_a_listener(): + # `list_sessions` reports it to the GUI table regardless of whether + # anyone subscribed to the event, so the wrapper is always installed. + host = _host() + host.create_session_offer() + _fire(_FakeHost.instances[0], "on_authenticated") + assert host.list_sessions()[0]["connected_at"] + + +def test_a_pending_viewer_is_reported_with_the_id_read_at_fire_time(): + seen = [] + host = _host(on_pending_viewer=lambda sid, vid: seen.append((sid, vid))) + session_id, _ = host.create_session_offer() + # The id does not exist when the wrapper is built -- only when a viewer + # actually knocks -- so the wrapper has to look the host up on each call. + _FakeHost.instances[0].pending_viewer_id = "viewer-9" + _fire(_FakeHost.instances[0], "on_pending_viewer") + assert seen == [(session_id, "viewer-9")] + + +def test_a_pending_viewer_from_a_session_that_just_left_reports_none(): + seen = [] + host = _host(on_pending_viewer=lambda sid, vid: seen.append((sid, vid))) + session_id, _ = host.create_session_offer() + wrapper = _FakeHost.instances[0].kwargs["on_pending_viewer"] + host.stop_session(session_id) + wrapper() + assert seen == [(session_id, None)] + + +def test_no_pending_callback_means_no_wrapper_is_installed(): + host = _host() + host.create_session_offer() + assert _FakeHost.instances[0].kwargs["on_pending_viewer"] is None + + +@pytest.mark.parametrize("kwarg,wrapper_key", [ + ("on_session_state", "on_state_change"), + ("on_session_authenticated", "on_authenticated"), + ("on_pending_viewer", "on_pending_viewer"), +]) +def test_a_raising_gui_callback_never_reaches_the_session(kwarg, wrapper_key): + # These fire on the asyncio thread; an exception escaping one of them + # would kill the event loop that every other session shares. + def _boom(*_args): + raise RuntimeError("Qt widget already deleted") + + host = _host(**{kwarg: _boom}) + host.create_session_offer() + wrapper = _FakeHost.instances[0].kwargs[wrapper_key] + if wrapper_key == "on_state_change": + wrapper("connected") + else: + wrapper() diff --git a/test/unit_test/headless/test_remote_desktop_trust_and_quality.py b/test/unit_test/headless/test_remote_desktop_trust_and_quality.py new file mode 100644 index 00000000..7d153ac4 --- /dev/null +++ b/test/unit_test/headless/test_remote_desktop_trust_and_quality.py @@ -0,0 +1,459 @@ +"""The remote-desktop trust store, address book and quality controller. + +These four modules decide who is allowed to drive this machine unattended +(`trust_list`), whether the host answering is the host that answered last time +(`fingerprint`), what the viewer offers to reconnect to (`address_book`), and +how hard the encoder is pushed when the link degrades (`adaptive_bitrate`). +They are ordinary Python -- a JSON file, a lock and some arithmetic -- but +until the `[webrtc]` extra became part of the measured install they were +imported by nothing on any CI square, because `adaptive_bitrate` reaches +`webrtc_stats` and the whole subsystem raised ImportError at module level. + +What these tests are for: every one of them describes a decision the code makes +in the operator's absence. A trust entry that silently loses its label on +re-add, a known-hosts file that throws on a truncated write, a fingerprint +comparison that is case-sensitive against an SDP that is not, a downscale that +fires on a single dropped packet -- each is invisible until someone is +connected from another building. +""" +import json + +import pytest + +from je_auto_control.utils.remote_desktop.address_book import AddressBook +from je_auto_control.utils.remote_desktop.adaptive_bitrate import ( + AdaptiveBitrateController, +) +from je_auto_control.utils.remote_desktop.fingerprint import ( + FingerprintMismatchError, KnownHosts, extract_dtls_fingerprint, + fingerprint_for_display, load_or_create_host_fingerprint, + verify_dtls_fingerprint, +) +from je_auto_control.utils.remote_desktop.trust_list import TrustList +from je_auto_control.utils.remote_desktop.webrtc_stats import StatsSnapshot + + +# === The host's own fingerprint ============================================= + +def test_host_fingerprint_is_created_once_and_then_reused(tmp_path): + """First call mints 64 hex chars; later calls return the same string.""" + target = tmp_path / "nested" / "host_fingerprint" + first = load_or_create_host_fingerprint(target) + assert len(first) == 64 + assert int(first, 16) >= 0 # hex, not arbitrary text + assert load_or_create_host_fingerprint(target) == first + assert target.read_text(encoding="utf-8").strip() == first + + +def test_a_truncated_fingerprint_file_is_replaced_not_trusted(tmp_path): + """A half-written file must not become this host's identity.""" + target = tmp_path / "host_fingerprint" + target.write_text("deadbeef", encoding="utf-8") + minted = load_or_create_host_fingerprint(target) + assert len(minted) == 64 + assert minted != "deadbeef" + + +def test_an_unwritable_fingerprint_path_still_returns_one(tmp_path): + """Persistence is best-effort: the session gets an identity regardless.""" + blocker = tmp_path / "blocker" + blocker.write_text("not a directory", encoding="utf-8") + minted = load_or_create_host_fingerprint(blocker / "sub" / "fp") + assert len(minted) == 64 + + +# === The viewer's known-hosts map =========================================== + +def test_known_hosts_survives_a_restart(tmp_path): + path = tmp_path / "known_hosts.json" + first = KnownHosts(path) + first.remember("host-a", "a" * 64) + first.remember_dtls_fingerprint("host-a", "AB:CD") + first.touch("host-a") + + second = KnownHosts(path) + assert second.fingerprint_for("host-a") == "a" * 64 + assert second.dtls_fingerprint_for("host-a") == "AB:CD" + assert second.last_seen("host-a") is not None + + +def test_remembering_one_fingerprint_keeps_the_other(tmp_path): + """The app-layer and DTLS fingerprints are stored side by side.""" + hosts = KnownHosts(tmp_path / "known_hosts.json") + hosts.remember_dtls_fingerprint("host-a", "AB:CD") + hosts.remember("host-a", "a" * 64) + assert hosts.dtls_fingerprint_for("host-a") == "AB:CD" + hosts.remember_dtls_fingerprint("host-a", "EF:01") + assert hosts.fingerprint_for("host-a") == "a" * 64 + assert hosts.dtls_fingerprint_for("host-a") == "EF:01" + + +def test_a_legacy_plain_string_entry_is_migrated_on_load(tmp_path): + """Files written before the DTLS fingerprint existed still open.""" + path = tmp_path / "known_hosts.json" + path.write_text(json.dumps({"host-a": "a" * 64}), encoding="utf-8") + hosts = KnownHosts(path) + assert hosts.fingerprint_for("host-a") == "a" * 64 + assert hosts.dtls_fingerprint_for("host-a") is None + + +@pytest.mark.parametrize("payload", ["{not json", json.dumps(["a", "b"]), + json.dumps({"host-a": 7})]) +def test_an_unreadable_known_hosts_file_opens_empty(tmp_path, payload): + """A corrupt store must not stop the viewer from connecting at all.""" + path = tmp_path / "known_hosts.json" + path.write_text(payload, encoding="utf-8") + assert KnownHosts(path).list_entries() == {} + + +def test_forget_reports_whether_anything_was_removed(tmp_path): + hosts = KnownHosts(tmp_path / "known_hosts.json") + hosts.remember("host-a", "a" * 64) + assert hosts.forget("host-a") is True + assert hosts.forget("host-a") is False + assert hosts.fingerprint_for("host-a") is None + + +def test_list_entries_hands_back_copies(tmp_path): + """A caller mutating the report must not rewrite the trust store.""" + hosts = KnownHosts(tmp_path / "known_hosts.json") + hosts.remember("host-a", "a" * 64) + hosts.list_entries()["host-a"]["app_fp"] = "tampered" + assert hosts.fingerprint_for("host-a") == "a" * 64 + + +# === Showing and comparing fingerprints ===================================== + +def test_display_form_groups_a_full_fingerprint_into_fours(): + grouped = fingerprint_for_display("ab" * 32) + assert grouped.count(":") == 15 + assert grouped.replace(":", "") == "ab" * 32 + + +@pytest.mark.parametrize("value, expected", [("", ""), ("short", "short")]) +def test_display_form_passes_through_what_it_cannot_group(value, expected): + assert fingerprint_for_display(value) == expected + + +_SDP = ( + "v=0\r\n" + "a=fingerprint:sha-1 11:22:33\r\n" + "m=video 9 UDP/TLS/RTP/SAVPF 96\r\n" + "a=fingerprint:SHA-256 ab:cd:ef:01\r\n" +) + + +def test_the_requested_algorithm_is_the_one_extracted(): + """An offer carries several; picking the wrong line pins the wrong cert.""" + assert extract_dtls_fingerprint(_SDP, "sha-256") == "AB:CD:EF:01" + assert extract_dtls_fingerprint(_SDP, "sha-1") == "11:22:33" + assert extract_dtls_fingerprint(_SDP, "sha-384") is None + + +def test_a_non_string_offer_yields_no_fingerprint(): + assert extract_dtls_fingerprint(None) is None + + +@pytest.mark.parametrize("expected", ["ab:cd:ef:01", "ABCDEF01", + "AB:CD:EF:01"]) +def test_verification_accepts_either_spelling_of_the_same_value(expected): + verify_dtls_fingerprint(_SDP, expected) + + +def test_verification_rejects_a_different_certificate(): + with pytest.raises(FingerprintMismatchError, match="mismatch"): + verify_dtls_fingerprint(_SDP, "00:11:22:33") + + +def test_verification_fails_closed_when_the_offer_pins_nothing(): + with pytest.raises(FingerprintMismatchError, match="no sha-256"): + verify_dtls_fingerprint("v=0\r\n", "AB:CD") + + +# === The unattended-access trust list ======================================= + +def test_a_trusted_viewer_is_still_trusted_after_a_restart(tmp_path): + path = tmp_path / "trusted_viewers.json" + TrustList(path).add("viewer-1", label="office laptop") + reopened = TrustList(path) + assert reopened.is_trusted("viewer-1") is True + assert reopened.list_entries()[0]["label"] == "office laptop" + + +def test_re_adding_keeps_the_original_label_and_first_seen_time(tmp_path): + """Re-authenticating must not quietly erase what the operator typed.""" + trust = TrustList(tmp_path / "trusted_viewers.json") + trust.add("viewer-1", label="office laptop") + added_at = trust.list_entries()[0]["added_at"] + trust.add("viewer-1") + entry = trust.list_entries()[0] + assert entry["label"] == "office laptop" + assert entry["added_at"] == added_at + assert len(trust.list_entries()) == 1 + + +def test_a_new_label_replaces_the_old_one(tmp_path): + trust = TrustList(tmp_path / "trusted_viewers.json") + trust.add("viewer-1", label="old") + trust.add("viewer-1", label="new") + assert trust.list_entries()[0]["label"] == "new" + + +def test_touch_records_use_only_for_a_viewer_already_trusted(tmp_path): + trust = TrustList(tmp_path / "trusted_viewers.json") + trust.touch("stranger") + assert trust.list_entries() == [] + trust.add("viewer-1") + assert trust.list_entries()[0]["last_used"] is None + trust.touch("viewer-1") + assert trust.list_entries()[0]["last_used"] is not None + + +def test_touch_preserves_last_used_across_a_re_add(tmp_path): + trust = TrustList(tmp_path / "trusted_viewers.json") + trust.add("viewer-1") + trust.touch("viewer-1") + last_used = trust.list_entries()[0]["last_used"] + trust.add("viewer-1") + assert trust.list_entries()[0]["last_used"] == last_used + + +@pytest.mark.parametrize("viewer_id", ["", None, 7]) +def test_an_empty_or_non_string_viewer_id_is_refused(tmp_path, viewer_id): + """A blank id would trust every viewer that fails to send one.""" + trust = TrustList(tmp_path / "trusted_viewers.json") + with pytest.raises(ValueError): + trust.add(viewer_id) + + +@pytest.mark.parametrize("viewer_id", [None, 7, b"viewer-1"]) +def test_a_non_string_id_is_never_trusted(tmp_path, viewer_id): + assert TrustList(tmp_path / "trusted_viewers.json").is_trusted( + viewer_id) is False + + +def test_remove_and_clear_report_and_persist(tmp_path): + path = tmp_path / "trusted_viewers.json" + trust = TrustList(path) + trust.add("viewer-1") + trust.add("viewer-2") + assert trust.remove("viewer-1") is True + assert trust.remove("viewer-1") is False + trust.clear() + assert TrustList(path).list_entries() == [] + + +@pytest.mark.parametrize("payload", ["{not json", json.dumps(["a"]), + json.dumps({"viewers": "not a list"}), + json.dumps({"viewers": [1, {}]})]) +def test_an_unreadable_trust_list_opens_empty(tmp_path, payload): + """Fail closed: a damaged file trusts nobody rather than everybody.""" + path = tmp_path / "trusted_viewers.json" + path.write_text(payload, encoding="utf-8") + assert TrustList(path).list_entries() == [] + + +# === The viewer's address book ============================================== + +def test_upsert_refreshes_the_matching_entry_instead_of_appending(tmp_path): + book = AddressBook(tmp_path / "book.json") + book.upsert(host_id="h1", server_url="ws://a", label="desk") + first_used = book.list_entries()[0]["last_used"] + book.upsert(host_id="h1", server_url="ws://a", mac_address="00:11:22") + entries = book.list_entries() + assert len(entries) == 1 + assert entries[0]["label"] == "desk" # blank label does not erase + assert entries[0]["mac_address"] == "00:11:22" + assert entries[0]["last_used"] >= first_used + + +def test_the_same_host_on_a_different_url_is_a_different_entry(tmp_path): + book = AddressBook(tmp_path / "book.json") + book.upsert(host_id="h1", server_url="ws://a") + book.upsert(host_id="h1", server_url="ws://b") + assert len(book.list_entries()) == 2 + + +@pytest.mark.parametrize("host_id, server_url", [("", "ws://a"), ("h1", "")]) +def test_an_incomplete_target_is_refused(tmp_path, host_id, server_url): + book = AddressBook(tmp_path / "book.json") + with pytest.raises(ValueError): + book.upsert(host_id=host_id, server_url=server_url) + + +def test_tags_are_stripped_deduplicated_and_sorted(tmp_path): + book = AddressBook(tmp_path / "book.json") + book.upsert(host_id="h1", server_url="ws://a") + book.upsert(host_id="h2", server_url="ws://b") + book.set_tags(host_id="h1", server_url="ws://a", tags=[" lab ", "", "prod"]) + book.set_tags(host_id="h2", server_url="ws://b", tags=["prod", None]) + assert book.list_entries()[0]["tags"] == ["lab", "prod"] + # A JSON `null` is dropped, not stringified into a tag named "None". + assert book.all_tags() == ["lab", "prod"] + + +def test_setting_tags_on_an_unknown_target_changes_nothing(tmp_path): + book = AddressBook(tmp_path / "book.json") + book.upsert(host_id="h1", server_url="ws://a") + book.set_tags(host_id="nope", server_url="ws://z", tags=["x"]) + assert book.all_tags() == [] + + +def test_toggle_favorite_reports_the_new_state_and_persists(tmp_path): + path = tmp_path / "book.json" + book = AddressBook(path) + book.upsert(host_id="h1", server_url="ws://a") + assert book.toggle_favorite(host_id="h1", server_url="ws://a") is True + assert book.toggle_favorite(host_id="h1", server_url="ws://a") is False + assert book.toggle_favorite(host_id="nope", server_url="ws://z") is False + assert AddressBook(path).list_entries()[0]["favorite"] is False + + +def test_remove_and_clear_report_and_persist_for_the_book(tmp_path): + path = tmp_path / "book.json" + book = AddressBook(path) + book.upsert(host_id="h1", server_url="ws://a") + assert book.remove(host_id="h1", server_url="ws://a") is True + assert book.remove(host_id="h1", server_url="ws://a") is False + book.upsert(host_id="h2", server_url="ws://b") + book.clear() + assert AddressBook(path).list_entries() == [] + + +@pytest.mark.parametrize("payload", ["{not json", json.dumps(["a"]), + json.dumps({"entries": [1, {}, { + "host_id": 7}]})]) +def test_an_unreadable_address_book_opens_empty(tmp_path, payload): + path = tmp_path / "book.json" + path.write_text(payload, encoding="utf-8") + assert AddressBook(path).list_entries() == [] + + +# === The adaptive quality controller ======================================== + +class _FakeTrack: + """The one thing the controller drives: a track with a target FPS.""" + + def __init__(self, fps: int = 20) -> None: + self.fps = fps + self.calls = [] + + def set_target_fps(self, value: int) -> None: + self.calls.append(value) + self.fps = value + + +def _feed(controller, count, **fields): + for _ in range(count): + controller.on_stats(StatsSnapshot(**fields)) + + +def test_a_single_bad_sample_does_not_move_the_encoder(): + """Hysteresis: one dropped packet is noise, not a degraded link.""" + track = _FakeTrack(20) + controller = AdaptiveBitrateController(track) + _feed(controller, 1, packet_loss_pct=9.0) + assert track.calls == [] + assert controller.current_fps == 20 + + +def test_two_consecutive_lossy_samples_step_the_rate_down(): + track = _FakeTrack(20) + controller = AdaptiveBitrateController(track) + _feed(controller, 2, packet_loss_pct=9.0) + assert track.calls == [16] + + +def test_a_latency_spike_downscales_the_same_way_loss_does(): + track = _FakeTrack(20) + controller = AdaptiveBitrateController(track) + _feed(controller, 2, rtt_ms=400.0) + assert track.calls == [16] + + +def test_a_good_sample_between_two_bad_ones_resets_the_streak(): + track = _FakeTrack(20) + controller = AdaptiveBitrateController(track) + _feed(controller, 1, packet_loss_pct=9.0) + _feed(controller, 1, packet_loss_pct=3.0) # neither down nor up + _feed(controller, 1, packet_loss_pct=9.0) + assert track.calls == [] + + +def test_the_rate_never_falls_through_the_floor(): + track = _FakeTrack(7) + controller = AdaptiveBitrateController(track, floor_fps=6) + _feed(controller, 2, packet_loss_pct=9.0) + assert track.fps == 6 + _feed(controller, 2, packet_loss_pct=9.0) + assert track.calls == [6] # already at the floor: no second call + + +def test_climbing_back_takes_four_clean_samples_and_stops_at_the_ceiling(): + track = _FakeTrack(12) + controller = AdaptiveBitrateController(track, max_fps=14) + _feed(controller, 3, packet_loss_pct=0.1) + assert track.calls == [] + _feed(controller, 1, packet_loss_pct=0.1) + assert track.calls == [14] # +4 would be 16; the ceiling wins + _feed(controller, 4, packet_loss_pct=0.1) + assert track.calls == [14] # at the ceiling: nothing further + + +def test_sustained_latency_keeps_stepping_down_however_clean_the_link(): + """Loss near zero does not license a climb while RTT stays bad.""" + track = _FakeTrack(12) + controller = AdaptiveBitrateController(track, max_fps=20) + _feed(controller, 8, packet_loss_pct=0.1, rtt_ms=400.0) + assert track.calls == [8, 5] # the RTT rule downscales every two samples + + +def test_exceeding_the_hard_bitrate_cap_steps_down_on_the_first_sample(): + """A configured cap is a promise to the network, not a trend to confirm.""" + track = _FakeTrack(20) + controller = AdaptiveBitrateController(track, max_bitrate_kbps=1000) + _feed(controller, 1, bitrate_kbps=4000.0, packet_loss_pct=0.1) + assert track.calls == [16] + + +def test_the_hard_cap_resets_the_quality_streaks(): + """A cap breach mid-climb must not count towards the next upscale.""" + track = _FakeTrack(12) + controller = AdaptiveBitrateController(track, max_fps=20, + max_bitrate_kbps=1000) + _feed(controller, 3, packet_loss_pct=0.1) + _feed(controller, 1, bitrate_kbps=4000.0) + assert track.calls == [8] + _feed(controller, 3, packet_loss_pct=0.1) + assert track.calls == [8] # the streak restarted from zero + + +def test_a_bitrate_under_the_cap_leaves_the_hard_path_alone(): + track = _FakeTrack(20) + controller = AdaptiveBitrateController(track, max_bitrate_kbps=5000) + _feed(controller, 1, bitrate_kbps=1000.0, packet_loss_pct=9.0) + assert track.calls == [] # quality path only, and it needs two + + +def test_disabling_the_controller_freezes_the_rate(): + track = _FakeTrack(20) + controller = AdaptiveBitrateController(track) + controller.set_enabled(False) + _feed(controller, 6, packet_loss_pct=9.0) + assert track.calls == [] + controller.set_enabled(True) + _feed(controller, 2, packet_loss_pct=9.0) + assert track.calls == [16] + + +def test_a_controller_with_no_track_reports_no_rate(): + controller = AdaptiveBitrateController(_FakeTrack(20)) + controller._track = None # the teardown state the host leaves behind + controller.on_stats(StatsSnapshot(packet_loss_pct=9.0)) + assert controller.current_fps == 0 + + +def test_a_stats_snapshot_crosses_the_json_boundary(): + snapshot = StatsSnapshot(rtt_ms=12.5, fps=30.0, bitrate_kbps=900.0, + packet_loss_pct=0.2, jitter_ms=3.0) + assert json.loads(json.dumps(snapshot.to_dict()))["rtt_ms"] == 12.5 diff --git a/test/unit_test/headless/test_rest_route_sweep.py b/test/unit_test/headless/test_rest_route_sweep.py new file mode 100644 index 00000000..cbd52a45 --- /dev/null +++ b/test/unit_test/headless/test_rest_route_sweep.py @@ -0,0 +1,225 @@ +"""Every REST route, called the way its own OpenAPI document says to call it. + +`rest_handlers` is the third registry of the same shape as the MCP tool table +and the `AC_*` dispatch table: thirty-odd functions that take a decoded request, +call one headless function, and return `(status, payload)` for the dispatcher to +write out as JSON. Its module docstring says the handlers are "pure ... trivial +to unit-test without an HTTP layer", and the existing REST tests go through the +HTTP layer instead, so on a headless runner most of them reach a handler only to +watch it fall into its own `except` and answer 500. The happy path -- the branch +a real client actually gets -- was checked by nobody. + +The arguments come from `rest_openapi.build_openapi_spec()`, which is built from +`_ENDPOINT_METADATA` in a different module from the handlers. That matters for +the same reason the MCP sweep reads `_factories.py`: a sweep that took its +arguments from the handler it is testing could not fail. Here it also buys a +contract test for free -- a route the document does not describe, or a documented +route nothing serves, is a defect either way, and `test_the_route_table_and_the_ +document_agree` says so by name. + +Two things are asserted about every route: + +* **It answers.** No route may raise: the dispatcher writes the handler's return + value straight into the socket, so an exception is a dropped HTTP response and + a client that hangs until it times out. This holds for a well-formed request + and for the empty one an unhelpful client sends. +* **The answer is JSON with an HTTP status.** `(int, dict)`, a status in the + 100-599 range, and a payload `json.dumps` accepts. + +The callee is replaced by a stub grown from its own return annotation, exactly +as in the other two sweeps -- see `_contract_sweep`. +""" +import json +from typing import Any, Dict, List, Optional, Tuple +from urllib.parse import urlencode + +import pytest + +from headless._contract_sweep import ( + contract_stubs, install_stubs, is_serialisable, sample_value, +) +from je_auto_control.utils.rest_api import rest_server +from je_auto_control.utils.rest_api.rest_handlers import RouteContext +from je_auto_control.utils.rest_api.rest_openapi import build_openapi_spec + +_SPEC = build_openapi_spec() +_JSON = "application/json" + +# Served by the dispatcher itself rather than from the two route tables, +# because none of them answers with JSON: Prometheus text, the dashboard HTML +# and the Swagger UI page. They are in the document because a client still has +# to be told they exist. +_NOT_JSON_ROUTES = {("GET", "/dashboard"), ("GET", "/docs"), ("GET", "/metrics")} + + +@pytest.fixture(autouse=True) +def _in_a_directory_of_its_own(tmp_path, monkeypatch): + """Run every sweep case in an empty directory. + + An adapter whose callee is a class builds the real object out of the + client's own arguments, and some of those objects are stores: a + checkpoint store handed the sample file path creates a SQLite database + where it stands. In the repository that leaves files behind and makes one + case depend on whether another ran first; here each case gets a directory + nobody else can see. + """ + monkeypatch.chdir(tmp_path) + + +def _routes() -> List[Tuple[str, str, Any]]: + """Every `(method, path, handler)` the dispatcher can reach.""" + return ([("GET", path, handler) + for path, handler in rest_server._GET_ROUTES.items()] + + [("POST", path, handler) + for path, handler in rest_server._POST_ROUTES.items()]) + + +ROUTES = sorted(_routes(), key=lambda route: (route[0], route[1])) +_IDS = [f"{method} {path}" for method, path, _ in ROUTES] + + +def _operation(method: str, path: str) -> Dict[str, Any]: + return _SPEC["paths"].get(path, {}).get(method.lower(), {}) + + +def _query_for(operation: Dict[str, Any]) -> str: + """Build a query string from the operation's declared parameters.""" + pairs = [(parameter["name"], + sample_value(parameter.get("schema") or {}, parameter["name"])) + for parameter in operation.get("parameters", []) + if parameter.get("in") == "query"] + return urlencode([(name, str(value)) for name, value in pairs]) + + +def _body_for(operation: Dict[str, Any]) -> Optional[Any]: + """Build a request body from the operation's declared requestBody.""" + content = (operation.get("requestBody") or {}).get("content") or {} + schema = (content.get(_JSON) or {}).get("schema") + if not schema: + return None + return {name: sample_value(spec, name) + for name, spec in (schema.get("properties") or {}).items()} + + +def _assert_answers_json(result: Any, label: str) -> None: + """A handler's return value is what the dispatcher writes to the socket.""" + assert isinstance(result, tuple) and len(result) == 2, ( + f"{label} returned {result!r}, not (status, payload)") + status, payload = result + assert isinstance(status, int) and 100 <= status <= 599, ( + f"{label} returned {status!r}, which is not an HTTP status") + assert isinstance(payload, dict), ( + f"{label} returned a {type(payload).__name__} payload; the dispatcher " + "writes a JSON object") + assert is_serialisable(payload), ( + f"{label} returned a payload json.dumps cannot encode") + + +# === The document and the table have to describe the same API =============== + +def test_the_route_table_and_the_document_agree(): + """A route nobody documents, or a documented route nobody serves.""" + documented = {(method.upper(), path) + for path, item in _SPEC["paths"].items() for method in item} + routed = {(method, path) for method, path, _ in ROUTES} + assert routed - documented == set(), ( + "routes the OpenAPI document does not describe: " + f"{sorted(routed - documented)}") + assert documented - routed == _NOT_JSON_ROUTES, ( + "documented routes nothing serves, or a non-JSON route that grew a " + f"JSON handler: {sorted(documented - routed - _NOT_JSON_ROUTES)}") + + +def test_every_route_is_reachable_from_the_dispatcher(): + """The two tables are what `handle_request` looks in; neither may be empty.""" + assert len(rest_server._GET_ROUTES) > 10 + assert len(rest_server._POST_ROUTES) > 5 + assert not set(rest_server._GET_ROUTES) & set(rest_server._POST_ROUTES), ( + "a path in both tables would resolve by method alone; nothing here " + "expects that today") + + +# === Every route answers, twice ============================================== + +@pytest.mark.parametrize("method, path, handler", ROUTES, ids=_IDS) +def test_a_documented_request_gets_a_json_answer(method, path, handler, + monkeypatch): + """Called as the document says, every route answers in JSON.""" + operation = _operation(method, path) + stubs = contract_stubs(handler) + if stubs is not None: + install_stubs(monkeypatch, stubs) + context = RouteContext(query=_query_for(operation), + body=_body_for(operation), + client_ip="127.0.0.1") + _assert_answers_json(handler(context), f"{method} {path}") + + +@pytest.mark.parametrize("method, path, handler", ROUTES, ids=_IDS) +def test_an_empty_request_gets_a_json_answer(method, path, handler, + monkeypatch): + """A client that sends nothing gets a status, not a dropped connection.""" + stubs = contract_stubs(handler) + if stubs is not None: + install_stubs(monkeypatch, stubs) + context = RouteContext(query="", body=None, client_ip="127.0.0.1") + _assert_answers_json(handler(context), f"{method} {path}") + + +@pytest.mark.parametrize("method, path, handler", ROUTES, ids=_IDS) +def test_a_body_of_the_wrong_shape_is_rejected_not_raised(method, path, + handler, + monkeypatch): + """Bodies arrive from the network; a list where a dict was expected is + the client's mistake to be told about, not the server's to crash on.""" + stubs = contract_stubs(handler) + if stubs is not None: + install_stubs(monkeypatch, stubs) + context = RouteContext(query="limit=not-a-number&n=not-a-number", + body=["not", "an", "object"], + client_ip="127.0.0.1") + _assert_answers_json(handler(context), f"{method} {path}") + + +# === The sweep has to keep finding things ==================================== + +def test_the_sweep_covers_the_whole_table(): + """A broken matcher would sweep nothing and still look green.""" + assert len(ROUTES) >= 30, f"only {len(ROUTES)} routes discovered" + stubbable = [route for route in ROUTES if contract_stubs(route[2])] + assert len(stubbable) > len(ROUTES) // 3, ( + f"only {len(stubbable)} of {len(ROUTES)} handlers could be stubbed " + "from their callee's annotations -- the typing contract or the " + "matcher has drifted") + + +# === The pieces the routes are built from ==================================== + +def test_query_parsing_takes_the_first_value_and_falls_back(): + context = RouteContext(query="limit=5&limit=9&other=x", body=None, + client_ip="127.0.0.1") + assert context.query_first("limit") == "5" + assert context.query_first("missing") is None + assert context.query_first("missing", "fallback") == "fallback" + assert context.query_params()["limit"] == ["5", "9"] + + +def test_a_route_without_a_query_string_parses_to_nothing(): + context = RouteContext(query="", body=None, client_ip="127.0.0.1") + assert context.query_params() == {} + assert context.query_first("limit", "100") == "100" + + +def test_health_is_the_one_route_that_needs_no_token(): + """A liveness probe must not need the bearer token the server minted.""" + assert rest_server._PUBLIC_PATHS == frozenset({"/health"}) + status, payload = rest_server.handle_health( + RouteContext(query="", body=None, client_ip="127.0.0.1")) + assert (status, payload) == (200, {"status": "ok"}) + + +def test_the_published_document_is_json_and_describes_this_api(): + spec = json.loads(json.dumps(_SPEC)) + assert spec["openapi"].startswith("3.") + assert spec["info"]["title"] + assert spec["paths"] diff --git a/test/unit_test/headless/test_webrtc_file_transfer.py b/test/unit_test/headless/test_webrtc_file_transfer.py new file mode 100644 index 00000000..911e036b --- /dev/null +++ b/test/unit_test/headless/test_webrtc_file_transfer.py @@ -0,0 +1,329 @@ +"""The file channel: what a remote peer can and cannot write to this disk. + +`webrtc_files` reassembles a viewer's upload into a file under the host's inbox. +The name in the envelope is chosen by the peer, so this module is a filesystem +write driven by remote input, and its docstring says the defence out loud: +"incoming filenames are stripped of any directory components to defeat path +traversal". Nothing tested it -- the module reaches `webrtc_transport`, which +raises ImportError at module level without aiortc, so it could not be imported +on any CI square until the `[webrtc]` extra joined the measured install. + +Both halves are exercised here, and then wired to each other: the sender's +output is fed straight into the receiver, which is the only way to check that +the two agree about the protocol their shared docstring describes. +""" +import json + +import pytest + +from je_auto_control.utils.remote_desktop import webrtc_files +from je_auto_control.utils.remote_desktop.webrtc_files import ( + FileTransferError, FileTransferReceiver, FileTransferSender, +) + + +class _Bridge: + """`call_soon` on the real bridge runs the call on the loop thread.""" + + def call_soon(self, callback, *args): + callback(*args) + + +class _Channel: + """A DataChannel that keeps what was sent.""" + + def __init__(self): + self.messages = [] + + def send(self, message): + self.messages.append(message) + + +@pytest.fixture +def bridge(monkeypatch): + monkeypatch.setattr(webrtc_files, "get_bridge", _Bridge) + + +def _begin(name, size, transfer_id="t1"): + return json.dumps({"type": "file_begin", "name": name, "size": size, + "transfer_id": transfer_id}) + + +def _end(transfer_id="t1"): + return json.dumps({"type": "file_end", "transfer_id": transfer_id}) + + +def _receiver(tmp_path): + return FileTransferReceiver(tmp_path / "inbox") + + +# === What the peer may name the file ======================================== + +@pytest.mark.parametrize("name, written_as", [ + ("report.txt", "report.txt"), + ("../../../etc/passwd", "passwd"), + ("a/b/c/nested.bin", "nested.bin"), + ("./visible.txt", "visible.txt"), + ("trailing/", "trailing"), +]) +def test_a_directory_in_the_name_never_escapes_the_inbox(bridge, tmp_path, + name, written_as): + receiver = _receiver(tmp_path) + receiver.handle_message(_begin(name, 2)) + receiver.handle_message(b"hi") + receiver.handle_message(_end()) + inbox = tmp_path / "inbox" + assert [entry.name for entry in inbox.iterdir()] == [written_as] + assert (inbox / written_as).read_bytes() == b"hi" + + +@pytest.mark.parametrize("name", ["", ".", "..", "bad\x00name", + "pipe|name", "star*name", "quest?name", + "ltname", 'quote"name']) +def test_a_name_that_cannot_be_made_safe_is_refused(bridge, tmp_path, name): + receiver = _receiver(tmp_path) + errors = [] + receiver.handle_message(_begin(name, 2), on_error=errors.append) + assert errors, f"{name!r} was accepted" + assert list((tmp_path / "inbox").iterdir()) == [] + + +def test_a_non_string_name_is_refused(bridge, tmp_path): + receiver = _receiver(tmp_path) + errors = [] + receiver.handle_message( + json.dumps({"type": "file_begin", "name": None, "size": 1}), + on_error=errors.append) + assert errors + + +@pytest.mark.parametrize("size", [-1, 5 * 1024 * 1024 * 1024]) +def test_an_impossible_size_is_refused_before_any_file_is_opened( + bridge, tmp_path, size): + receiver = _receiver(tmp_path) + errors = [] + receiver.handle_message(_begin("big.bin", size), on_error=errors.append) + assert errors + assert list((tmp_path / "inbox").iterdir()) == [] + + +# === The protocol =========================================================== + +def test_a_transfer_arrives_in_pieces_and_reports_progress(bridge, tmp_path): + receiver = _receiver(tmp_path) + progress, done = [], [] + receiver.handle_message(_begin("split.bin", 6)) + receiver.handle_message(b"abc", on_progress=lambda n, total: + progress.append((n, total))) + receiver.handle_message(b"def", on_progress=lambda n, total: + progress.append((n, total))) + receiver.handle_message(_end(), on_done=done.append) + assert progress == [(3, 6), (6, 6)] + assert done and done[0].read_bytes() == b"abcdef" + + +@pytest.mark.parametrize("chunk", [b"raw", bytearray(b"raw"), + memoryview(b"raw")]) +def test_a_chunk_arrives_however_the_transport_spells_bytes(bridge, tmp_path, + chunk): + receiver = _receiver(tmp_path) + receiver.handle_message(_begin("x.bin", 3)) + receiver.handle_message(chunk) + receiver.handle_message(_end()) + assert (tmp_path / "inbox" / "x.bin").read_bytes() == b"raw" + + +def test_a_chunk_with_no_transfer_open_is_dropped(bridge, tmp_path): + """A late chunk from an aborted transfer must not create a file.""" + receiver = _receiver(tmp_path) + receiver.handle_message(b"orphan") + assert list((tmp_path / "inbox").iterdir()) == [] + + +def test_an_end_with_no_transfer_open_is_ignored(bridge, tmp_path): + receiver = _receiver(tmp_path) + done = [] + receiver.handle_message(_end(), on_done=done.append) + assert done == [] + + +def test_a_second_begin_is_refused_and_takes_the_first_with_it(bridge, + tmp_path): + """One transfer per channel: the module says so, and it fails closed.""" + receiver = _receiver(tmp_path) + errors = [] + receiver.handle_message(_begin("first.bin", 10)) + receiver.handle_message(b"partial") + receiver.handle_message(_begin("second.bin", 10), on_error=errors.append) + assert errors == ["transfer already in progress"] + # The partial first file is removed rather than left as a truncated one. + assert list((tmp_path / "inbox").iterdir()) == [] + + +def test_an_abort_from_the_sender_removes_the_partial_file(bridge, tmp_path): + receiver = _receiver(tmp_path) + errors = [] + receiver.handle_message(_begin("partial.bin", 100)) + receiver.handle_message(b"half") + receiver.handle_message(json.dumps({"type": "file_abort"}), + on_error=errors.append) + assert errors == ["aborted by sender"] + assert list((tmp_path / "inbox").iterdir()) == [] + + +def test_an_abort_with_nothing_in_flight_is_harmless(bridge, tmp_path): + receiver = _receiver(tmp_path) + receiver.handle_message(json.dumps({"type": "file_abort"})) + assert list((tmp_path / "inbox").iterdir()) == [] + + +def test_a_malformed_envelope_is_reported_not_raised(bridge, tmp_path): + receiver = _receiver(tmp_path) + errors = [] + receiver.handle_message("{not json", on_error=errors.append) + assert errors and "bad envelope" in errors[0] + + +def test_an_unknown_envelope_type_is_ignored(bridge, tmp_path): + receiver = _receiver(tmp_path) + receiver.handle_message(json.dumps({"type": "file_pause"})) + assert list((tmp_path / "inbox").iterdir()) == [] + + +def test_a_message_that_is_neither_text_nor_bytes_is_ignored(bridge, tmp_path): + receiver = _receiver(tmp_path) + receiver.handle_message(42) + assert list((tmp_path / "inbox").iterdir()) == [] + + +def test_the_inbox_is_created_on_construction(bridge, tmp_path): + inbox = tmp_path / "deep" / "inbox" + FileTransferReceiver(inbox) + assert inbox.is_dir() + + +# === The sending half ======================================================= + +def test_a_sender_needs_a_channel(): + with pytest.raises(ValueError): + FileTransferSender(None) + + +def test_sending_a_path_that_is_not_a_file_is_refused(bridge, tmp_path): + sender = FileTransferSender(_Channel()) + with pytest.raises(FileTransferError, match="not a file"): + sender.send(tmp_path / "missing.txt") + with pytest.raises(FileTransferError, match="not a file"): + sender.send(tmp_path) + + +def test_a_send_is_a_begin_then_chunks_then_an_end(bridge, tmp_path): + source = tmp_path / "payload.bin" + source.write_bytes(b"0123456789") + channel = _Channel() + progress = [] + FileTransferSender(channel).send(source, chunk_size=4, + on_progress=lambda n, total: + progress.append((n, total))) + begin = json.loads(channel.messages[0]) + assert begin["type"] == "file_begin" + assert begin["name"] == "payload.bin" + assert begin["size"] == 10 + assert channel.messages[1:-1] == [b"0123", b"4567", b"89"] + end = json.loads(channel.messages[-1]) + assert end["type"] == "file_end" + assert end["transfer_id"] == begin["transfer_id"] + assert progress == [(4, 10), (8, 10), (10, 10)] + + +def test_an_empty_file_is_a_begin_and_an_end_with_no_chunks(bridge, tmp_path): + source = tmp_path / "empty.bin" + source.write_bytes(b"") + channel = _Channel() + FileTransferSender(channel).send(source) + assert [json.loads(message)["type"] for message in channel.messages] == [ + "file_begin", "file_end"] + + +def test_the_remote_name_is_sanitised_before_it_leaves(bridge, tmp_path): + """The sender is the other side's remote peer; it does not get to say + where its file lands either.""" + source = tmp_path / "payload.bin" + source.write_bytes(b"x") + channel = _Channel() + FileTransferSender(channel).send(source, remote_name="a/b/../evil.txt") + assert json.loads(channel.messages[0])["name"] == "evil.txt" + + +def test_a_read_failure_mid_transfer_aborts_the_receiver_too(bridge, tmp_path): + source = tmp_path / "payload.bin" + source.write_bytes(b"0123456789") + + class _Failing: + def __init__(self, real): + self._real = real + self._reads = 0 + + def read(self, size): + self._reads += 1 + if self._reads > 1: + raise OSError("device disappeared") + return self._real.read(size) + + def __enter__(self): + return self + + def __exit__(self, *exc): + self._real.close() + return False + + real_open = type(source).open + + def _open(self, *args, **kwargs): + handle = real_open(self, *args, **kwargs) + return _Failing(handle) if self == source else handle + + channel = _Channel() + monkey = pytest.MonkeyPatch() + monkey.setattr(type(source), "open", _open) + try: + with pytest.raises(FileTransferError, match="read failed"): + FileTransferSender(channel).send(source, chunk_size=4) + finally: + monkey.undo() + assert json.loads(channel.messages[-1])["type"] == "file_abort" + + +# === The two halves agree =================================================== + +def test_a_file_survives_a_round_trip_through_the_protocol(bridge, tmp_path): + """Neither side's view of the protocol is asserted from the other's code.""" + source = tmp_path / "round.bin" + source.write_bytes(bytes(range(256)) * 40) # 10,240 bytes + receiver = _receiver(tmp_path) + done = [] + + class _Wired: + def send(self, message): + receiver.handle_message(message, on_done=done.append) + + FileTransferSender(_Wired()).send(source, remote_name="landed.bin", + chunk_size=1024) + assert done, "the receiver never completed the transfer" + assert done[0].name == "landed.bin" + assert done[0].read_bytes() == source.read_bytes() + + +def test_an_aborted_send_leaves_nothing_behind_on_the_receiver(bridge, + tmp_path): + receiver = _receiver(tmp_path) + + class _Wired: + def send(self, message): + receiver.handle_message(message) + + channel = _Wired() + receiver.handle_message(_begin("stale.bin", 100)) + receiver.handle_message(b"partial") + channel.send(json.dumps({"type": "file_abort"})) + assert list((tmp_path / "inbox").iterdir()) == [] diff --git a/test/unit_test/headless/test_webrtc_host_auth.py b/test/unit_test/headless/test_webrtc_host_auth.py new file mode 100644 index 00000000..eb6e947d --- /dev/null +++ b/test/unit_test/headless/test_webrtc_host_auth.py @@ -0,0 +1,381 @@ +"""Everything between a viewer's first `auth` message and its first keystroke. + +`ViewerAuthMixin` is the half of the WebRTC host that decides whether a peer +that just connected may drive this machine: the token check, the two +auto-approve paths (trust list, IP whitelist), the manual accept/reject the GUI +drives, and the grace period that closes a peer which never authenticates. It +is the security boundary of the whole remote-desktop subsystem, and until the +`[webrtc]` extra became part of the measured install nothing imported it on any +CI square -- `webrtc_transport` raises ImportError at module level without +aiortc, so the module could not even be loaded to be tested. + +The mixin's own docstring lists what it needs from the host it is mixed into +(`_token`, `_trust_list`, `_ip_whitelist`, `_authenticated`, `_send_ctrl`, +`_spawn_bg`, `_async_stop`, ...). `_Host` below supplies exactly that list and +nothing else, so a mixin that starts reaching for something new fails here +rather than depending on whatever `WebRTCDesktopHost` happens to also own. + +The two collaborators that reach outside the process are replaced: the asyncio +bridge, whose `call_soon` would otherwise queue the work onto a loop no test is +running, and the audit log, which writes to the user's real +`~/.je_auto_control`. +""" +import asyncio + +import pytest + +from je_auto_control.utils.remote_desktop import webrtc_host_auth as auth_module +from je_auto_control.utils.remote_desktop.permissions import SessionPermissions +from je_auto_control.utils.remote_desktop.trust_list import TrustList +from je_auto_control.utils.remote_desktop.webrtc_host_auth import ViewerAuthMixin + + +class _ImmediateBridge: + """The asyncio bridge, minus the loop: run the callback where it is made.""" + + def __init__(self) -> None: + self.deferred = [] + + def call_soon(self, callback) -> None: + self.deferred.append(callback) + callback() + + +class _AuditLog: + def __init__(self) -> None: + self.events = [] + + def log(self, event_type, **fields) -> None: + self.events.append((event_type, fields)) + + +class _Host(ViewerAuthMixin): + """A host with exactly the attributes the mixin's docstring asks for.""" + + def __init__(self, *, token="secret", trust_list=None, ip_whitelist=None, + remote_ip=None, permissions=None, on_pending_viewer=None): + self._token = token + self._trust_list = trust_list + self._ip_whitelist = list(ip_whitelist or []) + self._permissions = permissions or SessionPermissions.full_control() + self._remote_ip = remote_ip + self._authenticated = False + self._has_pending_viewer = False + self._pending_viewer_id = None + self._auth_deadline_handle = None + self._on_authenticated = None + self._on_pending_viewer = on_pending_viewer + self.sent = [] + self.stopped = 0 + self.spawned = [] + + def _send_ctrl(self, message): + self.sent.append(message) + + def _spawn_bg(self, coroutine): + coroutine.close() # never awaited here; do not leak the frame + self.spawned.append(coroutine) + return coroutine + + async def _async_stop(self): + self.stopped += 1 + + def types_sent(self): + return [message["type"] for message in self.sent] + + +@pytest.fixture +def bridge(monkeypatch): + """Replace the asyncio bridge and the audit log for the whole module. + + A loop is installed but never run, which is the shape the real bridge + presents to this code: ``call_soon`` work happens on the loop thread, so + ``asyncio.get_event_loop()`` resolves there, while anything handed to + ``call_later`` is still only scheduled when the test looks at it. + """ + loop = asyncio.new_event_loop() + asyncio.set_event_loop(loop) + immediate = _ImmediateBridge() + log = _AuditLog() + monkeypatch.setattr(auth_module, "get_bridge", lambda: immediate) + monkeypatch.setattr(auth_module, "default_audit_log", lambda: log) + monkeypatch.setattr(auth_module, "load_or_create_host_fingerprint", + lambda: "f" * 64) + immediate.audit = log + immediate.loop = loop + yield immediate + asyncio.set_event_loop(None) + loop.close() + + +# === The token is the gate ================================================== + +@pytest.mark.parametrize("token", ["wrong", "", None, 7, ["secret"]]) +def test_a_wrong_or_malformed_token_never_authenticates(bridge, token): + """The token arrives from the network; only an equal string may pass.""" + host = _Host(token="secret") + host._handle_auth({"token": token, "viewer_id": "v1"}) + assert host._authenticated is False + assert host.types_sent() == ["auth_fail"] + + +def test_a_missing_token_field_is_a_wrong_token(bridge): + host = _Host(token="secret") + host._handle_auth({"viewer_id": "v1"}) + assert host.types_sent() == ["auth_fail"] + + +def test_a_rejected_viewer_is_audited_and_then_disconnected(bridge): + host = _Host(token="secret", remote_ip="10.0.0.9") + host._handle_auth({"token": "wrong", "viewer_id": "v1"}) + assert [event for event, _ in bridge.audit.events] == ["auth_fail"] + fields = bridge.audit.events[0][1] + assert fields["viewer_id"] == "v1" + assert "10.0.0.9" in fields["detail"] + # `_schedule_close_after_fail` was handed to the bridge, so the peer is + # on its way out rather than left holding an unauthenticated channel. + assert bridge.deferred + + +def test_the_right_token_with_no_prompt_hook_approves_immediately(bridge): + """Headless use has no GUI to ask; the token is then the whole check.""" + host = _Host(token="secret") + host._handle_auth({"token": "secret", "viewer_id": "v1"}) + assert host._authenticated is True + assert host.types_sent() == ["auth_ok"] + + +def test_the_right_token_with_a_prompt_hook_waits_for_a_person(bridge): + asked = [] + host = _Host(token="secret", on_pending_viewer=lambda: asked.append(True)) + host._handle_auth({"token": "secret", "viewer_id": "v1"}) + assert asked == [True] + assert host.has_pending_viewer is True + assert host._authenticated is False + assert host.types_sent() == [] + + +def test_a_prompt_hook_that_throws_leaves_the_viewer_pending(bridge): + """A broken GUI callback must not authenticate the peer by accident.""" + def explode(): + raise RuntimeError("no window") + + host = _Host(token="secret", on_pending_viewer=explode) + host._handle_auth({"token": "secret", "viewer_id": "v1"}) + assert host._authenticated is False + assert host.has_pending_viewer is True + + +def test_a_non_string_viewer_id_is_recorded_as_absent(bridge): + host = _Host(token="secret", on_pending_viewer=lambda: None) + host._handle_auth({"token": "secret", "viewer_id": 7}) + assert host.pending_viewer_id is None + + +# === Auto-approve by trust list ============================================= + +def test_a_trusted_viewer_skips_the_prompt(bridge, tmp_path): + trust = TrustList(tmp_path / "trusted.json") + trust.add("v1", label="office laptop") + asked = [] + host = _Host(token="secret", trust_list=trust, + on_pending_viewer=lambda: asked.append(True)) + host._handle_auth({"token": "secret", "viewer_id": "v1"}) + assert asked == [] + assert host._authenticated is True + assert trust.list_entries()[0]["last_used"] is not None + + +def test_an_untrusted_viewer_still_gets_the_prompt(bridge, tmp_path): + trust = TrustList(tmp_path / "trusted.json") + trust.add("someone-else") + asked = [] + host = _Host(token="secret", trust_list=trust, + on_pending_viewer=lambda: asked.append(True)) + host._handle_auth({"token": "secret", "viewer_id": "v1"}) + assert asked == [True] + assert host._authenticated is False + + +def test_a_trust_list_that_throws_does_not_auto_approve(bridge): + """Fail closed: an unreadable trust store trusts nobody.""" + class _Broken: + def is_trusted(self, viewer_id): + raise OSError("disk gone") + + asked = [] + host = _Host(token="secret", trust_list=_Broken(), + on_pending_viewer=lambda: asked.append(True)) + host._handle_auth({"token": "secret", "viewer_id": "v1"}) + assert host._authenticated is False + assert asked == [True] + + +def test_trusting_the_pending_viewer_adds_it_and_approves(bridge, tmp_path): + trust = TrustList(tmp_path / "trusted.json") + host = _Host(token="secret", trust_list=trust, + on_pending_viewer=lambda: None) + host._handle_auth({"token": "secret", "viewer_id": "v1"}) + host.trust_pending_viewer(label="lab machine") + assert trust.is_trusted("v1") is True + assert trust.list_entries()[0]["label"] == "lab machine" + assert host._authenticated is True + + +def test_trusting_with_no_trust_list_still_approves(bridge): + host = _Host(token="secret", on_pending_viewer=lambda: None) + host._handle_auth({"token": "secret", "viewer_id": "v1"}) + host.trust_pending_viewer() + assert host._authenticated is True + + +# === Auto-approve by IP whitelist =========================================== + +@pytest.mark.parametrize("remote_ip, allowed", [ + ("10.0.0.5", True), + ("10.0.1.5", False), + ("192.168.1.7", True), + ("::1", False), +]) +def test_the_whitelist_matches_by_network_not_by_string(bridge, remote_ip, + allowed): + asked = [] + host = _Host(token="secret", remote_ip=remote_ip, + ip_whitelist=["10.0.0.0/24", " 192.168.1.7 "], + on_pending_viewer=lambda: asked.append(True)) + host._handle_auth({"token": "secret", "viewer_id": "v1"}) + assert host._authenticated is allowed + assert asked == ([] if allowed else [True]) + + +def test_an_ipv6_peer_matches_an_ipv6_whitelist(bridge): + host = _Host(token="secret", remote_ip="fd00::5", + ip_whitelist=["fd00::/8"], on_pending_viewer=lambda: None) + host._handle_auth({"token": "secret", "viewer_id": "v1"}) + assert host._authenticated is True + + +@pytest.mark.parametrize("whitelist", [["not-a-cidr"], ["10.0.0.0/99"], []]) +def test_an_unparseable_whitelist_entry_is_skipped_not_fatal(bridge, + whitelist): + host = _Host(token="secret", remote_ip="10.0.0.5", + ip_whitelist=whitelist, on_pending_viewer=lambda: None) + host._handle_auth({"token": "secret", "viewer_id": "v1"}) + assert host._authenticated is False + + +@pytest.mark.parametrize("remote_ip", [None, "", "not-an-address"]) +def test_a_peer_with_no_usable_address_never_matches(bridge, remote_ip): + host = _Host(token="secret", remote_ip=remote_ip, + ip_whitelist=["0.0.0.0/0"], on_pending_viewer=lambda: None) + host._handle_auth({"token": "secret", "viewer_id": "v1"}) + assert host._authenticated is False + + +# === What approval and rejection actually send ============================== + +def test_approval_tells_the_viewer_what_it_may_do(bridge): + host = _Host(token="secret", + permissions=SessionPermissions.view_only(), + on_pending_viewer=lambda: None) + host._handle_auth({"token": "secret", "viewer_id": "v1"}) + host.approve_pending_viewer() + message = host.sent[-1] + assert message["type"] == "auth_ok" + assert message["read_only"] is True + assert message["permissions"]["allow_input"] is False + assert message["fingerprint"] == "f" * 64 + + +def test_approval_is_audited_and_cancels_the_grace_period(bridge): + class _Handle: + cancelled = False + + def cancel(self): + self.cancelled = True + + handle = _Handle() + host = _Host(token="secret", remote_ip="10.0.0.9", + on_pending_viewer=lambda: None) + host._auth_deadline_handle = handle + host._handle_auth({"token": "secret", "viewer_id": "v1"}) + host.approve_pending_viewer() + assert handle.cancelled is True + assert host._auth_deadline_handle is None + assert [event for event, _ in bridge.audit.events] == ["auth_ok"] + + +def test_the_authenticated_callback_runs_and_its_failure_is_contained(bridge): + called = [] + host = _Host(token="secret", on_pending_viewer=lambda: None) + host._on_authenticated = lambda: called.append(True) + host._handle_auth({"token": "secret", "viewer_id": "v1"}) + host.approve_pending_viewer() + assert called == [True] + + def explode(): + raise RuntimeError("gui gone") + + other = _Host(token="secret", on_pending_viewer=lambda: None) + other._on_authenticated = explode + other._handle_auth({"token": "secret", "viewer_id": "v2"}) + other.approve_pending_viewer() + assert other._authenticated is True + + +def test_approving_twice_does_not_re_send_auth_ok(bridge): + """The GUI can double-click; the viewer must not see two sessions open.""" + host = _Host(token="secret", on_pending_viewer=lambda: None) + host._handle_auth({"token": "secret", "viewer_id": "v1"}) + host.approve_pending_viewer() + host.approve_pending_viewer() + assert host.types_sent() == ["auth_ok"] + + +def test_rejecting_the_pending_viewer_fails_and_closes(bridge): + host = _Host(token="secret", on_pending_viewer=lambda: None) + host._handle_auth({"token": "secret", "viewer_id": "v1"}) + host.reject_pending_viewer() + assert host.types_sent() == ["auth_fail"] + assert host.has_pending_viewer is False + assert host._authenticated is False + + +# === The grace period ======================================================= + +def test_the_deadline_closes_a_peer_that_never_authenticated(bridge): + host = _Host(token="secret", on_pending_viewer=lambda: None) + host._enforce_auth_deadline() + assert host.spawned, "an unauthenticated peer must be stopped" + + +def test_the_deadline_leaves_an_authenticated_peer_alone(bridge): + host = _Host(token="secret") + host._handle_auth({"token": "secret", "viewer_id": "v1"}) + host._enforce_auth_deadline() + assert host.spawned == [] + + +# === The secure attention sequence ========================================== + +def test_a_successful_sas_is_reported_to_the_viewer(bridge, monkeypatch): + import je_auto_control.utils.remote_desktop.session_actions as actions + monkeypatch.setattr(actions, "send_secure_attention_sequence", + lambda: None) + host = _Host(token="secret") + host._handle_send_sas() + assert host.types_sent() == ["sas_ok"] + + +def test_a_platform_that_cannot_send_sas_says_so_rather_than_raising( + bridge, monkeypatch): + import je_auto_control.utils.remote_desktop.session_actions as actions + + def unsupported(): + raise RuntimeError("SendSAS is Windows-only") + + monkeypatch.setattr(actions, "send_secure_attention_sequence", unsupported) + host = _Host(token="secret") + host._handle_send_sas() + assert host.types_sent() == ["sas_fail"] + assert "Windows-only" in host.sent[-1]["error"] diff --git a/test/unit_test/headless/test_webrtc_host_channels.py b/test/unit_test/headless/test_webrtc_host_channels.py new file mode 100644 index 00000000..e76d7eb0 --- /dev/null +++ b/test/unit_test/headless/test_webrtc_host_channels.py @@ -0,0 +1,747 @@ +"""What a connected viewer can make the host do, and what it cannot. + +The session setup is in `test_webrtc_host_session.py`; this file is about +the four DataChannels that ride on it, and it is mostly about refusal. Every +message here arrives from the other end of the wire, so each handler is a +boundary with the same three questions behind it: + +* **Has this viewer authenticated?** The channels open with the + PeerConnection, which is *before* the token is checked -- so a peer that + never authenticates can still push bytes at every one of them. +* **Do the session permissions allow it?** `read_only` is a shorthand over + five flags the operator can flip mid-session from the GUI, and each + channel consults a different one: input, files, audio. +* **Is it flooding?** The token buckets exist for a viewer that is + authenticated and permitted and still sending 10,000 events a second. + +The inbox handlers get the most attention because they are the ones that +take a *name* from the viewer and turn it into a path on the host's disk. +`_safe_basename` is what stands between `../../.ssh/authorized_keys` and the +host's home directory, and these tests pin down that the host actually +routes through it -- on the listing, the fetch and the delete alike. + +`get_bridge` and the audit log are replaced (from +`headless._webrtc_doubles`): the first would queue work onto a loop no test +is running, the second writes to the real `~/.je_auto_control`. +""" +from __future__ import annotations + +import asyncio +import json + +import pytest + +from headless._webrtc_doubles import ( + AuditLog, Bridge, Channel, FakePeerConnection, MicReceiver, +) +from je_auto_control.utils.remote_desktop import webrtc_host as host_module +from je_auto_control.utils.remote_desktop.permissions import SessionPermissions +from je_auto_control.utils.remote_desktop.rate_limit import RateLimitConfig +from je_auto_control.utils.remote_desktop.webrtc_host import WebRTCDesktopHost + + +@pytest.fixture(autouse=True) +def bridge(monkeypatch): + fake = Bridge() + monkeypatch.setattr(host_module, "get_bridge", lambda: fake) + # `send_file` hands the work to `FileTransferSender`, which reads + # `get_bridge` out of `webrtc_files` -- patching it here alone left that + # path on the real bridge, so the chunks landed on a background event + # loop and the assertion read `sent[0]` before anything was in it. It + # passed wherever the loop won the race, which was every square until a + # loaded runner lost it. + from je_auto_control.utils.remote_desktop import webrtc_files + monkeypatch.setattr(webrtc_files, "get_bridge", lambda: fake) + return fake + + +@pytest.fixture(autouse=True) +def audit_log(monkeypatch): + log = AuditLog() + monkeypatch.setattr(host_module, "default_audit_log", lambda: log) + return log + + +@pytest.fixture +def host(tmp_path): + """An authenticated host with a control channel, and an empty inbox.""" + instance = WebRTCDesktopHost(token="secret", inbox_dir=tmp_path / "inbox") + channel = Channel() + instance._control_channel = channel + instance._wire_control_channel(channel) + instance._authenticated = True + return instance + + +def _sent(host_instance): + return [json.loads(text) for text in host_instance._control_channel.sent] + + +def _deliver(host_instance, payload): + host_instance._control_channel.fire("message", json.dumps(payload)) + + +def _revoke_files(host_instance): + """Drop to read-only, then forget the permissions envelope that sends.""" + host_instance.set_read_only(True) + host_instance._control_channel.sent.clear() + + +# --- the control envelope ----------------------------------------------------- + +@pytest.mark.parametrize("message", [ + b"\x00\x01", 42, None, ["input"], +]) +def test_a_non_text_control_message_is_ignored(host, message): + # The ctrl channel is text-only; binary on it is either a bug at the + # other end or someone probing. + host._control_channel.fire("message", message) + assert _sent(host) == [] + + +def test_a_malformed_envelope_is_dropped(host): + host._control_channel.fire("message", "{not json") + assert _sent(host) == [] + + +def test_a_json_scalar_is_not_an_envelope(host): + host._control_channel.fire("message", '"input"') + assert _sent(host) == [] + + +def test_an_unknown_message_type_is_ignored(host): + _deliver(host, {"type": "reboot_the_machine"}) + assert _sent(host) == [] + + +def test_an_envelope_with_no_type_is_ignored(host): + _deliver(host, {"payload": {"kind": "mouse"}}) + assert _sent(host) == [] + + +def test_an_unauthenticated_peer_can_only_send_auth(): + host = WebRTCDesktopHost(token="secret") + channel = Channel() + host._control_channel = channel + host._wire_control_channel(channel) + dispatched = [] + host._dispatch = dispatched.append + _deliver(host, {"type": "input", "payload": {"kind": "mouse"}}) + assert dispatched == [], "input before auth is not input, it is noise" + + +def test_an_unauthenticated_peer_reaches_the_token_check(): + host = WebRTCDesktopHost(token="secret") + channel = Channel() + host._control_channel = channel + host._wire_control_channel(channel) + _deliver(host, {"type": "auth", "token": "secret"}) + assert host.authenticated is True + + +def test_the_control_channel_open_event_is_survivable(host): + host._control_channel.fire("open") + + +def test_closing_the_control_channel_ends_the_session(host): + host._control_channel.fire("close") + assert host.authenticated is False + + +# --- input -------------------------------------------------------------------- + +def test_input_reaches_the_dispatcher(host): + dispatched = [] + host._dispatch = dispatched.append + _deliver(host, {"type": "input", "payload": {"kind": "mouse", "x": 3}}) + assert dispatched == [{"kind": "mouse", "x": 3}] + + +def test_input_is_refused_in_read_only_mode(host): + dispatched = [] + host._dispatch = dispatched.append + host.set_read_only(True) + _deliver(host, {"type": "input", "payload": {"kind": "mouse"}}) + assert dispatched == [] + + +def test_a_payload_that_is_not_a_dict_never_reaches_the_dispatcher(host): + dispatched = [] + host._dispatch = dispatched.append + _deliver(host, {"type": "input", "payload": "click"}) + assert dispatched == [] + + +def test_a_failing_dispatch_does_not_kill_the_channel(host): + # `dispatch_input` raises the whole AutoControl exception family plus + # OSError; anything escaping here would take the DataChannel bridge with + # it and end the session over one bad event. + def _boom(_payload): + raise ValueError("unknown key name") + + host._dispatch = _boom + _deliver(host, {"type": "input", "payload": {"kind": "key"}}) + _deliver(host, {"type": "input", "payload": {"kind": "key"}}) + + +def test_a_flood_of_input_is_dropped_and_audited(audit_log): + host = WebRTCDesktopHost( + token="secret", rate_limit=RateLimitConfig(input_burst=0), + ) + channel = Channel() + host._control_channel = channel + host._wire_control_channel(channel) + host._authenticated = True + dispatched = [] + host._dispatch = dispatched.append + _deliver(host, {"type": "input", "payload": {"kind": "mouse"}}) + assert dispatched == [] + assert [event for event, _ in audit_log.events] == ["rate_limit_input"] + + +def test_the_flood_audit_entry_is_written_once_per_window(audit_log): + # One log line per five-second window, not one per dropped event -- + # otherwise the flood is mirrored straight into the audit log. + host = WebRTCDesktopHost( + token="secret", rate_limit=RateLimitConfig(input_burst=0), + ) + channel = Channel() + host._control_channel = channel + host._wire_control_channel(channel) + host._authenticated = True + host._dispatch = lambda payload: None + for _ in range(5): + _deliver(host, {"type": "input", "payload": {"kind": "mouse"}}) + assert len(audit_log.events) == 1 + + +def test_an_audit_log_that_cannot_be_written_does_not_break_the_drop( + audit_log): + audit_log.error = OSError("disk full") + host = WebRTCDesktopHost( + token="secret", rate_limit=RateLimitConfig(input_burst=0), + ) + channel = Channel() + host._control_channel = channel + host._wire_control_channel(channel) + host._authenticated = True + host._dispatch = lambda payload: None + _deliver(host, {"type": "input", "payload": {"kind": "mouse"}}) + + +# --- the secure attention sequence -------------------------------------------- + +def test_send_sas_is_refused_in_read_only_mode(host, monkeypatch): + called = [] + monkeypatch.setattr( + "je_auto_control.utils.remote_desktop.session_actions" + ".send_secure_attention_sequence", + lambda: called.append(1), + ) + host.set_read_only(True) + _deliver(host, {"type": "send_sas"}) + assert called == [] + + +def test_send_sas_reaches_the_platform_call_when_permitted(host, monkeypatch): + called = [] + monkeypatch.setattr( + "je_auto_control.utils.remote_desktop.session_actions" + ".send_secure_attention_sequence", + lambda: called.append(1), + ) + _deliver(host, {"type": "send_sas"}) + assert called == [1] + assert _sent(host)[-1]["type"] == "sas_ok" + + +# --- annotations -------------------------------------------------------------- + +def test_an_annotation_reaches_the_gui_callback(): + seen = [] + host = WebRTCDesktopHost(token="secret", on_annotation=seen.append) + channel = Channel() + host._control_channel = channel + host._wire_control_channel(channel) + host._authenticated = True + _deliver(host, {"type": "annotate", "shape": "arrow"}) + assert seen == [{"type": "annotate", "shape": "arrow"}] + + +def test_an_annotation_with_no_listener_is_dropped(host): + _deliver(host, {"type": "annotate", "shape": "arrow"}) + + +def test_a_raising_annotation_callback_is_contained(): + def _boom(_data): + raise RuntimeError("Qt widget already deleted") + + host = WebRTCDesktopHost(token="secret", on_annotation=_boom) + channel = Channel() + host._control_channel = channel + host._wire_control_channel(channel) + host._authenticated = True + _deliver(host, {"type": "annotate"}) + + +# --- renegotiation ------------------------------------------------------------ + +def test_a_renegotiate_answer_without_a_connection_is_ignored(host): + _deliver(host, {"type": "renegotiate_answer", "sdp": "v=0\r\nanswer"}) + assert host._background_tasks == set() + + +def test_a_renegotiate_answer_with_no_sdp_is_ignored(host): + host._pc = object() + _deliver(host, {"type": "renegotiate_answer", "sdp": None}) + assert host._background_tasks == set() + + +def test_a_renegotiate_answer_is_applied_on_the_event_loop(host): + pc = FakePeerConnection() + + async def _drive(): + host._pc = pc + _deliver(host, {"type": "renegotiate_answer", "sdp": "v=0 answer"}) + # The handler runs on the DataChannel callback; applying the answer + # is awaited work, so it is spawned rather than run inline. + assert len(host._background_tasks) == 1 + await asyncio.gather(*host._background_tasks) + + asyncio.run(_drive()) + assert [d.sdp for d in pc.remote_descriptions] == ["v=0 answer"] + + +def test_applying_a_renegotiate_answer_resubscribes_the_viewer_media(host): + pc = FakePeerConnection() + host._pc = pc + asyncio.run(host._async_apply_renegotiate_answer("v=0\r\nanswer")) + assert [d.sdp for d in pc.remote_descriptions] == ["v=0\r\nanswer"] + + +def test_a_renegotiate_answer_that_aiortc_rejects_is_logged_not_raised(host): + pc = FakePeerConnection() + pc.remote_description_error = RuntimeError("invalid SDP") + host._pc = pc + asyncio.run(host._async_apply_renegotiate_answer("v=0\r\nnonsense")) + + +def test_applying_an_answer_without_a_connection_is_a_no_op(host): + asyncio.run(host._async_apply_renegotiate_answer("v=0\r\nanswer")) + + +# --- the mic channel ---------------------------------------------------------- + +def _wire_mic(host_instance): + channel = Channel("mic") + host_instance._wire_mic_channel(channel) + return channel + + +def test_mic_chunks_reach_the_receiver(host): + receiver = MicReceiver() + host._mic_receiver = receiver + channel = _wire_mic(host) + channel.fire("message", b"pcm") + assert receiver.chunks == [b"pcm"] + + +def test_mic_chunks_are_dropped_before_authentication(host): + receiver = MicReceiver() + host._mic_receiver = receiver + host._authenticated = False + _wire_mic(host).fire("message", b"pcm") + assert receiver.chunks == [] + + +def test_mic_chunks_are_dropped_when_nobody_is_listening(host): + # The channel exists from the moment the PeerConnection does; the + # receiver only exists once the operator turned the mic on. + _wire_mic(host).fire("message", b"pcm") + + +def test_mic_chunks_are_dropped_when_audio_is_not_permitted(host): + receiver = MicReceiver() + host._mic_receiver = receiver + host.set_permissions(SessionPermissions.none()) + _wire_mic(host).fire("message", b"pcm") + assert receiver.chunks == [] + + +def test_enabling_mic_receive_starts_one_receiver(host, monkeypatch): + monkeypatch.setattr( + "je_auto_control.utils.remote_desktop.webrtc_mic.MicUplinkReceiver", + MicReceiver, + ) + host.enable_mic_receive() + first = host._mic_receiver + host.enable_mic_receive() + assert host._mic_receiver is first + assert first.started + + +def test_disabling_mic_receive_stops_it(host): + receiver = MicReceiver() + host._mic_receiver = receiver + host.disable_mic_receive() + assert receiver.stopped + assert host._mic_receiver is None + + +def test_disabling_a_mic_that_was_never_enabled_is_a_no_op(host): + host.disable_mic_receive() + assert host._mic_receiver is None + + +def test_a_mic_that_fails_to_close_is_still_forgotten(host): + receiver = MicReceiver() + receiver.stop_error = OSError("stream already closed") + host._mic_receiver = receiver + host.disable_mic_receive() + assert host._mic_receiver is None + + +# --- the files channel -------------------------------------------------------- + +def _file_begin(name="report.txt", size=4): + return json.dumps({"type": "file_begin", "name": name, "size": size}) + + +def test_an_incoming_file_lands_in_the_inbox(host): + channel = Channel("files") + host._wire_files_channel(channel) + channel.fire("message", _file_begin()) + channel.fire("message", b"data") + channel.fire("message", json.dumps({"type": "file_end"})) + assert (host._files_receiver._inbox / "report.txt").read_bytes() == b"data" + + +def test_an_unauthenticated_peer_cannot_write_to_the_inbox(host): + host._authenticated = False + channel = Channel("files") + host._wire_files_channel(channel) + channel.fire("message", _file_begin()) + channel.fire("message", b"data") + assert list(host._files_receiver._inbox.iterdir()) == [] + + +def test_a_viewer_without_file_permission_cannot_write_to_the_inbox(host): + host.set_read_only(True) + channel = Channel("files") + host._wire_files_channel(channel) + channel.fire("message", _file_begin()) + assert list(host._files_receiver._inbox.iterdir()) == [] + + +def test_a_flood_of_transfers_is_dropped_and_audited(audit_log, tmp_path): + host = WebRTCDesktopHost( + token="secret", inbox_dir=tmp_path / "inbox", + rate_limit=RateLimitConfig(files_burst=0), + ) + host._authenticated = True + channel = Channel("files") + host._wire_files_channel(channel) + channel.fire("message", _file_begin()) + assert list(host._files_receiver._inbox.iterdir()) == [] + assert [event for event, _ in audit_log.events] == ["rate_limit_files"] + + +def test_chunks_of_an_allowed_transfer_are_not_rate_limited(tmp_path): + # The bucket counts transfers, not bytes: a file already accepted must + # not stall halfway through because its chunks exhausted the same + # bucket its envelope came out of. + host = WebRTCDesktopHost( + token="secret", inbox_dir=tmp_path / "inbox", + rate_limit=RateLimitConfig(files_per_minute=60.0, files_burst=1), + ) + host._authenticated = True + channel = Channel("files") + host._wire_files_channel(channel) + channel.fire("message", _file_begin(size=6)) + for chunk in (b"ab", b"cd", b"ef"): + channel.fire("message", chunk) + channel.fire("message", json.dumps({"type": "file_end"})) + assert (host._files_receiver._inbox / "report.txt").read_bytes() == b"abcdef" + + +def test_the_transfer_flood_audit_entry_is_written_once_per_window(audit_log, + tmp_path): + # Same rule as the input bucket: one line per five-second window, not + # one per refused transfer. + host = WebRTCDesktopHost( + token="secret", inbox_dir=tmp_path / "inbox", + rate_limit=RateLimitConfig(files_burst=0), + ) + host._authenticated = True + channel = Channel("files") + host._wire_files_channel(channel) + for _ in range(4): + channel.fire("message", _file_begin()) + assert len(audit_log.events) == 1 + + +def test_a_completed_transfer_is_audited_and_announced(host, audit_log): + seen = [] + host.set_file_received_callback(seen.append) + channel = Channel("files") + host._wire_files_channel(channel) + channel.fire("message", _file_begin()) + channel.fire("message", b"data") + channel.fire("message", json.dumps({"type": "file_end"})) + assert [event for event, _ in audit_log.events] == ["file_received"] + assert seen and seen[0].name == "report.txt" + + +def test_a_raising_file_callback_does_not_lose_the_file(host): + def _boom(_path): + raise RuntimeError("Qt widget already deleted") + + host.set_file_received_callback(_boom) + host._on_file_done(host._inbox_dir) + + +def test_an_audit_failure_still_lets_the_callback_run(host, audit_log): + audit_log.error = OSError("disk full") + seen = [] + host.set_file_received_callback(seen.append) + host._on_file_done("C:/inbox/report.txt") + assert seen == ["C:/inbox/report.txt"] + + +def test_pushing_a_file_before_a_viewer_connects_is_refused(host): + host._files_channel = None + with pytest.raises(RuntimeError, match="not connected"): + host.push_file("C:/report.txt") + + +def test_pushing_a_file_to_an_unauthenticated_peer_is_refused(host): + host._files_channel = Channel("files") + host._authenticated = False + with pytest.raises(RuntimeError, match="not connected"): + host.push_file("C:/report.txt") + + +def test_pushing_a_file_streams_it_over_the_files_channel(host, tmp_path): + source = tmp_path / "notes.txt" + source.write_bytes(b"hello") + host._files_channel = Channel("files") + host.push_file(str(source), remote_name="renamed.txt") + envelope = json.loads(host._files_channel.sent[0]) + assert envelope["type"] == "file_begin" + assert envelope["name"] == "renamed.txt" + + +# --- the usb channel ---------------------------------------------------------- + +def test_the_usb_channel_is_gated_on_auth_and_the_global_opt_in(host, + monkeypatch): + channel = Channel("usb") + host._wire_usb_channel(channel) + gate = host._usb_host._enabled + + monkeypatch.setattr( + "je_auto_control.utils.usb.passthrough.is_usb_passthrough_enabled", + lambda: False, + ) + assert gate() is False, "authenticated is not enough; it is opt-in" + + monkeypatch.setattr( + "je_auto_control.utils.usb.passthrough.is_usb_passthrough_enabled", + lambda: True, + ) + assert gate() is True + + host._authenticated = False + assert gate() is False, "the opt-in is not enough either" + + +def test_the_usb_session_is_built_with_a_default_deny_acl(host, monkeypatch): + import je_auto_control.utils.usb.passthrough as passthrough + + built = {} + + def _session(backend, acl=None, viewer_id=None): + built.update(backend=backend, acl=acl, viewer_id=viewer_id) + return "session" + + monkeypatch.setattr(passthrough, "UsbPassthroughSession", _session) + monkeypatch.setattr(passthrough, "default_passthrough_backend", + lambda: "backend") + channel = Channel("usb") + host._viewer_id = "viewer-3" + host._wire_usb_channel(channel) + assert host._usb_host._factory() == "session" + assert built["backend"] == "backend" + assert built["viewer_id"] == "viewer-3" + assert built["acl"] is not None, "never an unrestricted session" + + +# --- the inbox listing -------------------------------------------------------- + +def _inbox(host_instance): + return host_instance._ensure_files_receiver()._inbox + + +def test_listing_the_inbox_reports_name_size_and_mtime(host): + (_inbox(host) / "a.txt").write_bytes(b"12345") + _deliver(host, {"type": "list_inbox"}) + [response] = _sent(host) + assert response["type"] == "list_inbox_response" + [entry] = response["files"] + assert entry["name"] == "a.txt" + assert entry["size"] == 5 + assert "mtime" in entry + + +def test_listing_the_inbox_skips_directories(host): + (_inbox(host) / "sub").mkdir() + (_inbox(host) / "a.txt").write_bytes(b"1") + _deliver(host, {"type": "list_inbox"}) + assert [f["name"] for f in _sent(host)[0]["files"]] == ["a.txt"] + + +def test_listing_the_inbox_without_file_permission_is_refused(host): + (_inbox(host) / "a.txt").write_bytes(b"1") + _revoke_files(host) + _deliver(host, {"type": "list_inbox"}) + [response] = _sent(host) + assert response["files"] == [] + assert response["error"] == "files not permitted" + + +def test_an_unreadable_inbox_is_reported_as_an_error(host, monkeypatch): + receiver = host._ensure_files_receiver() + monkeypatch.setattr( + type(receiver._inbox), "iterdir", + lambda self: (_ for _ in ()).throw(OSError("permission denied")), + ) + _deliver(host, {"type": "list_inbox"}) + [response] = _sent(host) + assert response["files"] == [] + assert "permission denied" in response["error"] + + +def test_the_inbox_receiver_is_built_once_and_reused(host): + assert host._ensure_files_receiver() is host._ensure_files_receiver() + + +# --- fetching a file back ----------------------------------------------------- + +def test_requesting_a_file_pushes_it_over_the_files_channel(host): + (_inbox(host) / "a.txt").write_bytes(b"12345") + host._files_channel = Channel("files") + _deliver(host, {"type": "request_file", "name": "a.txt"}) + envelope = json.loads(host._files_channel.sent[0]) + assert (envelope["type"], envelope["name"]) == ("file_begin", "a.txt") + + +def test_requesting_a_missing_file_is_answered_not_ignored(host): + _deliver(host, {"type": "request_file", "name": "gone.txt"}) + [response] = _sent(host) + assert response == {"type": "request_file_response", "name": "gone.txt", + "ok": False, "error": "not found"} + + +def test_requesting_a_path_cannot_escape_the_inbox(host, tmp_path): + # The name arrives from the viewer; `..` in it must resolve to a + # basename inside the inbox rather than to the host's own files. + outside = tmp_path / "secret.txt" + outside.write_bytes(b"private") + _deliver(host, {"type": "request_file", "name": "../secret.txt"}) + [response] = _sent(host) + assert response["ok"] is False + assert response["name"] == "secret.txt", "sanitized to a bare basename" + + +def test_requesting_a_file_without_permission_is_silently_refused(host): + (_inbox(host) / "a.txt").write_bytes(b"1") + _revoke_files(host) + _deliver(host, {"type": "request_file", "name": "a.txt"}) + assert _sent(host) == [] + + +def test_a_request_with_a_non_string_name_is_ignored(host): + _deliver(host, {"type": "request_file", "name": {"path": "a.txt"}}) + assert _sent(host) == [] + + +def test_an_unusable_filename_is_reported_rather_than_raised(host): + _deliver(host, {"type": "request_file", "name": "co:n<>|.txt"}) + [response] = _sent(host) + assert response["type"] == "request_file_response" + assert response["ok"] is False + + +# --- deleting a file ---------------------------------------------------------- + +def test_deleting_an_inbox_file_removes_it(host): + target = _inbox(host) / "a.txt" + target.write_bytes(b"1") + _deliver(host, {"type": "delete_inbox_file", "name": "a.txt"}) + assert not target.exists() + assert _sent(host) == [{"type": "delete_inbox_response", "name": "a.txt", + "ok": True}] + + +def test_deleting_a_missing_file_is_answered_with_the_reason(host): + _deliver(host, {"type": "delete_inbox_file", "name": "gone.txt"}) + [response] = _sent(host) + assert response["ok"] is False + assert response["error"] + + +def test_deleting_without_permission_is_refused_with_a_reason(host): + target = _inbox(host) / "a.txt" + target.write_bytes(b"1") + host.set_read_only(True) + _deliver(host, {"type": "delete_inbox_file", "name": "a.txt"}) + assert target.exists() + assert _sent(host)[-1]["error"] == "files not permitted" + + +def test_a_delete_with_a_non_string_name_is_ignored(host): + _deliver(host, {"type": "delete_inbox_file", "name": 7}) + assert _sent(host) == [] + + +def test_deleting_a_path_cannot_escape_the_inbox(host, tmp_path): + outside = tmp_path / "secret.txt" + outside.write_bytes(b"private") + _deliver(host, {"type": "delete_inbox_file", "name": "../secret.txt"}) + assert outside.exists(), "the delete stayed inside the inbox" + assert _sent(host)[-1]["ok"] is False + + +# --- permissions and outbound sends ------------------------------------------- + +def test_changing_permissions_tells_the_viewer(host): + host.set_permissions(SessionPermissions.view_only()) + assert _sent(host)[-1] == { + "type": "permissions", + "value": SessionPermissions.view_only().to_dict(), + } + + +def test_read_only_is_derived_from_the_input_flag(host): + host.set_permissions(SessionPermissions.view_only()) + assert host.read_only is True + host.set_permissions(SessionPermissions.full_control()) + assert host.read_only is False + + +def test_sending_before_the_channel_exists_is_a_no_op(bridge): + host = WebRTCDesktopHost(token="secret") + host._send_ctrl({"type": "permissions"}) + assert bridge.deferred == [] + + +def test_a_channel_that_fails_mid_send_does_not_raise(host): + host._control_channel.send_error = OSError("channel closed") + host.set_permissions(SessionPermissions.view_only()) + + +def test_a_channel_closed_between_queue_and_send_is_a_no_op(host): + # `_send_ctrl` hops onto the asyncio loop, so the channel can go away + # between the two halves; `_safe_channel_send` re-checks for exactly that. + host._control_channel = None + host._safe_channel_send("{}") diff --git a/test/unit_test/headless/test_webrtc_host_media.py b/test/unit_test/headless/test_webrtc_host_media.py new file mode 100644 index 00000000..c91afc91 --- /dev/null +++ b/test/unit_test/headless/test_webrtc_host_media.py @@ -0,0 +1,347 @@ +"""Turning the viewer's camera and microphone on and off, asymmetrically. + +`MediaNegotiationMixin` exists because of one upstream fact its docstring +states: aiortc has no `removeTransceiver`. Enabling a viewer stream adds a +recvonly transceiver and re-offers; disabling can only mark the existing one +inactive and stop the receiver. The two directions therefore do different +things, and the slot each one reaches for is identified *by position* -- the +first video transceiver is the host's own outbound screen track, so the viewer's +is the second. Off-by-one there mutes the host's own screen share. + +Nothing imported this module on any CI square before the `[webrtc]` extra joined +the measured install; it reaches `webrtc_transport`, which raises ImportError at +module level without aiortc. + +`_pc`, the config and the spawn hook come from the host the mixin is mixed into, +and `_Host` supplies exactly the list the mixin's own docstring asks for. +""" +import asyncio + +import pytest + +from je_auto_control.utils.remote_desktop import webrtc_host_media as media +from je_auto_control.utils.remote_desktop.webrtc_host_media import ( + MediaNegotiationMixin, +) + + +class _Bridge: + def call_soon(self, callback): + callback() + + +class _Receiver: + def __init__(self, track=None): + self.track = track + + +class _Transceiver: + def __init__(self, kind, track=None, direction="sendrecv"): + self.kind = kind + self.receiver = _Receiver(track) if track is not None else None + self.direction = direction + + +class _PeerConnection: + def __init__(self, *transceivers): + self._transceivers = list(transceivers) + self.added = [] + + def getTransceivers(self): # noqa: N802 # reason: the aiortc name + return list(self._transceivers) + + def addTransceiver(self, kind, direction): # noqa: N802 # the aiortc name + self.added.append((kind, direction)) + self._transceivers.append(_Transceiver(kind, direction=direction)) + + +class _Config: + def __init__(self, *, accept_viewer_video=False, + accept_viewer_audio_opus=False): + self.accept_viewer_video = accept_viewer_video + self.accept_viewer_audio_opus = accept_viewer_audio_opus + + +class _Task: + def __init__(self): + self.cancelled = False + + def cancel(self): + self.cancelled = True + + +class _Host(MediaNegotiationMixin): + """A host with exactly the attributes the mixin's docstring asks for.""" + + def __init__(self, pc=None, config=None): + self._pc = pc + self._config = config or _Config() + self._viewer_video_task = None + self._opus_audio_receiver = None + self.sent = [] + self.spawned = [] + self.consumed = [] + self.opus_started = [] + + def _send_ctrl(self, message): + self.sent.append(message) + + def _spawn_bg(self, coroutine): + if asyncio.iscoroutine(coroutine): + coroutine.close() # never awaited here; do not leak the frame + self.spawned.append(coroutine) + return coroutine + + def _consume_viewer_video(self, track): + # The mixin calls this and hands the result to `_spawn_bg`, so the + # double records at call time and returns something awaitable — a + # coroutine body would not run until the task it never becomes. + self.consumed.append(track) + return self._noop() + + async def _noop(self): + return None + + def _start_opus_audio_receive(self, track): + self.opus_started.append(track) + + +@pytest.fixture(autouse=True) +def bridge(monkeypatch): + monkeypatch.setattr(media, "get_bridge", _Bridge) + + +# === Nothing happens without a peer connection ============================== + +@pytest.mark.parametrize("method", [ + "request_renegotiation", "enable_accept_viewer_video", + "enable_accept_viewer_audio_opus", "disable_accept_viewer_video", + "disable_accept_viewer_audio_opus", +]) +def test_every_entry_point_is_inert_before_a_connection_exists(method): + """These are GUI buttons; clicking one before connecting must not throw.""" + host = _Host(pc=None) + getattr(host, method)() + assert host.spawned == [] + assert host.sent == [] + + +# === Enabling adds capacity ================================================= + +def test_enabling_viewer_video_adds_a_recvonly_slot_and_renegotiates(): + pc = _PeerConnection(_Transceiver("video")) # the outbound screen + host = _Host(pc, _Config()) + host.enable_accept_viewer_video() + assert pc.added == [("video", "recvonly")] + assert host._config.accept_viewer_video is True + assert len(host.spawned) == 1 # the renegotiation + + +def test_enabling_twice_does_not_add_a_second_slot(): + """Two recvonly video slots would be two SDP m-lines nothing fills.""" + pc = _PeerConnection(_Transceiver("video")) + host = _Host(pc, _Config()) + host.enable_accept_viewer_video() + host.enable_accept_viewer_video() + assert pc.added == [("video", "recvonly")] + assert len(host.spawned) == 2 # but it re-offers again + + +def test_enabling_viewer_audio_adds_the_first_audio_slot(): + """There is no outbound audio slot to skip, so the first one is theirs.""" + pc = _PeerConnection(_Transceiver("video")) + host = _Host(pc, _Config()) + host.enable_accept_viewer_audio_opus() + assert pc.added == [("audio", "recvonly")] + assert host._config.accept_viewer_audio_opus is True + + +def test_enabling_audio_twice_does_not_add_a_second_slot(): + pc = _PeerConnection(_Transceiver("video")) + host = _Host(pc, _Config()) + host.enable_accept_viewer_audio_opus() + host.enable_accept_viewer_audio_opus() + assert pc.added == [("audio", "recvonly")] + + +# === Disabling can only deactivate ========================================== + +def test_disabling_viewer_video_leaves_the_hosts_own_track_alone(): + """The first video transceiver is the screen share; muting it is the bug + this test exists for.""" + outbound = _Transceiver("video") + inbound = _Transceiver("video") + host = _Host(_PeerConnection(outbound, inbound), + _Config(accept_viewer_video=True)) + host.disable_accept_viewer_video() + assert inbound.direction == "inactive" + assert outbound.direction == "sendrecv" + assert host._config.accept_viewer_video is False + + +def test_disabling_viewer_video_cancels_the_consume_task(): + task = _Task() + host = _Host(_PeerConnection(_Transceiver("video"), _Transceiver("video")), + _Config(accept_viewer_video=True)) + host._viewer_video_task = task + host.disable_accept_viewer_video() + assert task.cancelled is True + assert host._viewer_video_task is None + + +def test_disabling_viewer_video_with_no_inbound_slot_still_renegotiates(): + host = _Host(_PeerConnection(_Transceiver("video")), + _Config(accept_viewer_video=True)) + host.disable_accept_viewer_video() + assert len(host.spawned) == 1 + + +def test_a_transceiver_that_refuses_to_go_inactive_does_not_stop_the_rest(): + class _Stubborn: + kind = "video" + receiver = None + + @property + def direction(self): + return "sendrecv" + + @direction.setter + def direction(self, value): + raise RuntimeError("closed") + + task = _Task() + host = _Host(_PeerConnection(_Transceiver("video"), _Stubborn()), + _Config(accept_viewer_video=True)) + host._viewer_video_task = task + host.disable_accept_viewer_video() + assert task.cancelled is True + assert len(host.spawned) == 1 + + +def test_disabling_viewer_audio_deactivates_and_stops_the_receiver(): + class _Opus: + stopped = False + + def stop(self): + self.stopped = True + + audio = _Transceiver("audio") + receiver = _Opus() + host = _Host(_PeerConnection(_Transceiver("video"), audio), + _Config(accept_viewer_audio_opus=True)) + host._opus_audio_receiver = receiver + host.disable_accept_viewer_audio_opus() + assert audio.direction == "inactive" + assert receiver.stopped is True + assert host._opus_audio_receiver is None + + +def test_a_receiver_that_throws_on_stop_is_still_dropped(): + class _Opus: + def stop(self): + raise OSError("already gone") + + host = _Host(_PeerConnection(_Transceiver("audio")), + _Config(accept_viewer_audio_opus=True)) + host._opus_audio_receiver = _Opus() + host.disable_accept_viewer_audio_opus() + assert host._opus_audio_receiver is None + + +# === Re-subscribing after a renegotiation =================================== + +def test_the_viewer_video_track_is_picked_up_from_the_second_slot(): + track = object() + pc = _PeerConnection(_Transceiver("video", track=object()), + _Transceiver("video", track=track)) + host = _Host(pc, _Config(accept_viewer_video=True)) + host._maybe_resubscribe_viewer_video() + assert host.consumed == [track] + assert host._viewer_video_task is not None + + +def test_no_resubscribe_when_the_feature_is_off(): + pc = _PeerConnection(_Transceiver("video"), _Transceiver("video", + track=object())) + host = _Host(pc, _Config(accept_viewer_video=False)) + host._maybe_resubscribe_viewer_video() + assert host.consumed == [] + + +def test_no_resubscribe_when_a_task_is_already_running(): + pc = _PeerConnection(_Transceiver("video"), _Transceiver("video", + track=object())) + host = _Host(pc, _Config(accept_viewer_video=True)) + host._viewer_video_task = _Task() + host._maybe_resubscribe_viewer_video() + assert host.consumed == [] + + +def test_a_slot_with_no_track_yet_is_skipped(): + """A transceiver exists as soon as it is negotiated; the track arrives + later.""" + pc = _PeerConnection(_Transceiver("video"), _Transceiver("video")) + host = _Host(pc, _Config(accept_viewer_video=True)) + host._maybe_resubscribe_viewer_video() + assert host.consumed == [] + assert host._viewer_video_task is None + + +def test_the_viewer_audio_track_is_picked_up_from_any_audio_slot(): + track = object() + pc = _PeerConnection(_Transceiver("video", track=object()), + _Transceiver("audio", track=track)) + host = _Host(pc, _Config(accept_viewer_audio_opus=True)) + host._maybe_resubscribe_viewer_audio() + assert host.opus_started == [track] + + +def test_no_audio_resubscribe_when_a_receiver_is_already_running(): + pc = _PeerConnection(_Transceiver("audio", track=object())) + host = _Host(pc, _Config(accept_viewer_audio_opus=True)) + host._opus_audio_receiver = object() + host._maybe_resubscribe_viewer_audio() + assert host.opus_started == [] + + +# === The host-initiated offer =============================================== + +def test_renegotiation_sends_the_new_offer_over_the_control_channel( + monkeypatch): + class _Description: + sdp = "v=0\r\nfake offer\r\n" + + class _Negotiating(_PeerConnection): + localDescription = _Description() + + async def createOffer(self): # noqa: N802 # reason: the aiortc name + return _Description() + + async def setLocalDescription(self, offer): # noqa: N802 + self.local = offer + + async def _gathered(pc): + return None + + monkeypatch.setattr(media, "wait_for_ice_gathering", _gathered) + host = _Host(_Negotiating(), _Config()) + asyncio.run(host._async_renegotiate()) + assert host.sent == [{"type": "renegotiate_offer", + "sdp": "v=0\r\nfake offer\r\n"}] + + +def test_a_failed_offer_sends_nothing_rather_than_a_half_negotiation( + monkeypatch): + class _Broken(_PeerConnection): + async def createOffer(self): # noqa: N802 # reason: the aiortc name + raise RuntimeError("connection closed") + + host = _Host(_Broken(), _Config()) + asyncio.run(host._async_renegotiate()) + assert host.sent == [] + + +def test_renegotiating_without_a_connection_is_a_no_op(): + host = _Host(pc=None) + asyncio.run(host._async_renegotiate()) + assert host.sent == [] diff --git a/test/unit_test/headless/test_webrtc_host_session.py b/test/unit_test/headless/test_webrtc_host_session.py new file mode 100644 index 00000000..46cc5d68 --- /dev/null +++ b/test/unit_test/headless/test_webrtc_host_session.py @@ -0,0 +1,558 @@ +"""Building, renegotiating and tearing down a host session. + +`WebRTCDesktopHost` is the largest thing under `utils/remote_desktop` and, +until the `[webrtc]` extra joined the measured install, one of the least +covered: it reaches `webrtc_transport`, which raises ImportError at module +level without aiortc. Its two mixins already have tests +(`test_webrtc_host_auth.py`, `test_webrtc_host_media.py`); this file covers +the class those are mixed into. + +It splits along the seam the class itself has. Here: the session -- offer +construction, answer application, the connection-state handlers, and +teardown. The DataChannel traffic that rides on top is in +`test_webrtc_host_channels.py`. + +`RTCPeerConnection` and `ScreenVideoTrack` are replaced, from +`headless._webrtc_doubles`. A real one of either would open a screen +grabber and start STUN traffic on a CI runner, +and neither is what these tests are about: what matters is the *shape* of +the offer the host builds -- which transceivers it adds for which config, +which four DataChannels it opens, and in what order -- because that shape +is what the viewer's `_attach_viewer_screen_track` counts m-lines in. + +Teardown gets the same attention as setup for one reason: the host does +not own everything it holds. A relayed track belongs to `MultiViewerHost`, +and stopping it there would blank the screen for every *other* viewer. +""" +from __future__ import annotations + +import asyncio + +import pytest + +from headless._webrtc_doubles import ( + AuditLog, Bridge, FakePeerConnection, FrameTrack, HangingBridge, + Stoppable, Track, noop_ice_gathering, +) +from je_auto_control.utils.remote_desktop import webrtc_host as host_module +from je_auto_control.utils.remote_desktop.permissions import SessionPermissions +from je_auto_control.utils.remote_desktop.webrtc_host import WebRTCDesktopHost +from je_auto_control.utils.remote_desktop.webrtc_transport import WebRTCConfig + + +@pytest.fixture +def bridge(monkeypatch): + fake = Bridge() + monkeypatch.setattr(host_module, "get_bridge", lambda: fake) + return fake + + +@pytest.fixture(autouse=True) +def fake_peer_connection(monkeypatch): + """Replace the two collaborators that would touch a screen or a socket.""" + FakePeerConnection.instances = [] + monkeypatch.setattr(host_module, "RTCPeerConnection", FakePeerConnection) + monkeypatch.setattr(host_module, "ScreenVideoTrack", + lambda **kwargs: Track("video")) + monkeypatch.setattr(host_module, "wait_for_ice_gathering", + noop_ice_gathering) + yield + FakePeerConnection.instances = [] + + +@pytest.fixture(autouse=True) +def fake_audit_log(monkeypatch): + log = AuditLog() + monkeypatch.setattr(host_module, "default_audit_log", lambda: log) + return log + + +def _host(**kwargs) -> WebRTCDesktopHost: + kwargs.setdefault("token", "secret") + return WebRTCDesktopHost(**kwargs) + + +# --- construction ------------------------------------------------------------- + +def test_a_host_without_a_token_is_refused(): + with pytest.raises(ValueError, match="non-empty token"): + WebRTCDesktopHost(token="") + + +def test_read_only_shorthand_becomes_a_permission_set(): + assert _host(read_only=True).read_only is True + assert _host(read_only=True).permissions.allow_files is False + assert _host().read_only is False + + +def test_explicit_permissions_win_over_the_shorthand(): + permissions = SessionPermissions.view_only() + assert _host(read_only=False, permissions=permissions).permissions is ( + permissions + ) + + +def test_a_fresh_host_is_neither_authenticated_nor_connected(): + host = _host() + assert host.authenticated is False + assert host.connection_state == "closed", "no PeerConnection yet" + + +# --- building the offer ------------------------------------------------------- + +def test_create_offer_returns_the_local_sdp(bridge): + host = _host() + assert host.create_offer() == "v=0 local-sdp" + assert host.connection_state == "new" + + +def test_create_offer_asks_the_consent_callback_first(bridge): + seen = [] + + def _consent(peer_label): + seen.append(peer_label) + return True + + _host(offer_consent=_consent).create_offer(peer_label="Ops laptop") + assert seen == ["Ops laptop"] + + +def test_a_rejected_offer_never_builds_a_peer_connection(bridge): + host = _host(offer_consent=lambda peer: False) + with pytest.raises(PermissionError, match="rejected by consent"): + host.create_offer() + assert not FakePeerConnection.instances, "no capture, no ICE, nothing started" + + +def test_the_offer_carries_the_configured_ice_servers(bridge): + config = WebRTCConfig(ice_servers=["stun:example:3478"]) + _host(config=config).create_offer() + [pc] = FakePeerConnection.instances + assert [s.urls for s in pc.configuration.iceServers] == [ + "stun:example:3478", + ] + + +def test_the_offer_opens_the_four_data_channels_the_viewer_expects(bridge): + _host().create_offer() + [pc] = FakePeerConnection.instances + assert [c.label for c in pc.channels] == ["ctrl", "mic", "files", "usb"] + + +def test_the_screen_track_is_the_first_media_line(bridge): + # The viewer identifies the slot for its own screen by m-line order, so + # the host's outbound track has to be added before any recvonly slot. + config = WebRTCConfig(accept_viewer_video=True) + _host(config=config).create_offer() + [pc] = FakePeerConnection.instances + assert len(pc.tracks) == 1 + assert pc.transceivers == [("video", "recvonly")] + + +def test_no_inbound_slots_are_advertised_by_default(bridge): + _host().create_offer() + assert FakePeerConnection.instances[0].transceivers == [] + + +def test_accepting_viewer_audio_advertises_a_recvonly_audio_slot(bridge): + config = WebRTCConfig(accept_viewer_audio_opus=True) + _host(config=config).create_offer() + assert FakePeerConnection.instances[0].transceivers == [("audio", "recvonly")] + + +def test_an_external_track_is_used_as_is(bridge): + # `MultiViewerHost` hands each session a relay proxy of one shared + # capture; building a second grabber here would be a second screen read + # per viewer. + relayed = Track("video") + _host(external_video_track=relayed).create_offer() + assert FakePeerConnection.instances[0].tracks == [relayed] + + +def test_host_voice_attaches_a_microphone_track(bridge, monkeypatch): + mic = Track("audio") + monkeypatch.setattr( + "je_auto_control.utils.remote_desktop.webrtc_audio.OpusMicAudioTrack", + lambda: mic, + ) + _host(config=WebRTCConfig(host_voice=True)).create_offer() + assert mic in FakePeerConnection.instances[0].tracks + + +def test_a_host_with_no_microphone_still_gets_an_offer(bridge, monkeypatch): + # No input device is the normal state of a server; the screen share must + # not be lost because the mic could not be opened. + def _no_device(): + raise OSError("no input device") + + monkeypatch.setattr( + "je_auto_control.utils.remote_desktop.webrtc_audio.OpusMicAudioTrack", + _no_device, + ) + host = _host(config=WebRTCConfig(host_voice=True)) + assert host.create_offer() == "v=0 local-sdp" + assert host._host_voice_track is None + + +def test_a_second_offer_closes_the_first_peer_connection(bridge): + host = _host() + host.create_offer() + host.create_offer() + first, second = FakePeerConnection.instances + assert first.closed + assert not second.closed + + +# --- applying the answer ------------------------------------------------------ + +@pytest.mark.parametrize("answer", ["", " "]) +def test_an_empty_answer_is_refused_before_it_reaches_aiortc(bridge, answer): + host = _host() + host.create_offer() + with pytest.raises(ValueError, match="answer_sdp is empty"): + host.accept_answer(answer) + + +def test_accept_answer_before_create_offer_is_a_runtime_error(bridge): + with pytest.raises(RuntimeError, match="create_offer"): + _host().accept_answer("v=0 answer") + + +def test_accept_answer_applies_it_and_arms_the_auth_deadline(bridge): + host = _host() + host.create_offer() + host.accept_answer("v=0 answer") + [pc] = FakePeerConnection.instances + [description] = pc.remote_descriptions + assert (description.sdp, description.type) == ("v=0 answer", "answer") + assert host._auth_deadline_handle is not None, "the grace period is armed" + + +# --- connection state --------------------------------------------------------- + +def test_the_state_callback_sees_every_transition(bridge): + seen = [] + host = _host(on_state_change=seen.append) + host.create_offer() + pc = FakePeerConnection.instances[0] + for state in ("connecting", "connected", "failed"): + pc.connectionState = state + asyncio.run(pc.fire("connectionstatechange")) + assert seen == ["connecting", "connected", "failed"] + + +@pytest.mark.parametrize("state", ["failed", "closed", "disconnected"]) +def test_a_lost_connection_drops_the_authenticated_flag(bridge, state): + # Anything that reconnects starts a new session and must authenticate + # again; leaving the flag set would let a reused channel skip the token. + host = _host() + host.create_offer() + host._authenticated = True + pc = FakePeerConnection.instances[0] + pc.connectionState = state + asyncio.run(pc.fire("connectionstatechange")) + assert host.authenticated is False + + +def test_a_raising_state_callback_does_not_break_the_handler(bridge): + def _boom(_state): + raise RuntimeError("Qt widget already deleted") + + host = _host(on_state_change=_boom) + host.create_offer() + pc = FakePeerConnection.instances[0] + pc.connectionState = "connected" + asyncio.run(pc.fire("connectionstatechange")) + + +def test_connecting_snapshots_the_remote_ip_from_the_selected_pair(bridge): + host = _host() + host.create_offer() + pc = FakePeerConnection.instances[0] + pc.stats = _stats_with_selected_pair(ip="203.0.113.9") + pc.connectionState = "connected" + asyncio.run(pc.fire("connectionstatechange")) + assert host._remote_ip == "203.0.113.9" + + +def _stat(**fields): + return type("_Stat", (), fields)() + + +def _stats_with_selected_pair(*, ip=None, address=None, remote_id="remote-1", + include_remote=True): + stats = {"pair-1": _stat(type="candidate-pair", selected=True, + remoteCandidateId=remote_id)} + if include_remote: + fields = {} + if ip is not None: + fields["ip"] = ip + if address is not None: + fields["address"] = address + stats[remote_id] = _stat(**fields) + return stats + + +def test_the_remote_ip_falls_back_to_the_address_field(): + # aiortc renamed `ip` to `address` following the spec; both spellings + # turn up depending on the version installed. + report = _stats_with_selected_pair(address="198.51.100.4") + assert WebRTCDesktopHost._extract_remote_ip(report) == "198.51.100.4" + + +def test_an_unselected_candidate_pair_is_not_the_remote_peer(): + report = {"pair-1": _stat(type="candidate-pair", selected=False, + remoteCandidateId="remote-1")} + assert WebRTCDesktopHost._extract_remote_ip(report) is None + + +def test_a_dangling_candidate_reference_yields_no_ip(): + report = _stats_with_selected_pair(include_remote=False) + assert WebRTCDesktopHost._extract_remote_ip(report) is None + + +def test_a_candidate_with_neither_ip_nor_address_yields_no_ip(): + report = _stats_with_selected_pair() + assert WebRTCDesktopHost._extract_remote_ip(report) is None + + +def test_stats_that_cannot_be_read_leave_the_ip_unknown(bridge): + # The IP feeds the whitelist check and the audit log; failing to read it + # must not fail the connection. + host = _host() + host.create_offer() + FakePeerConnection.instances[0].stats_error = RuntimeError("pc already closed") + asyncio.run(host._snapshot_remote_ip()) + assert host._remote_ip is None + + +def test_snapshotting_without_a_peer_connection_is_a_no_op(): + asyncio.run(_host()._snapshot_remote_ip()) + + +# --- inbound viewer media ----------------------------------------------------- + +def test_an_inbound_video_track_starts_a_consume_task(bridge): + host = _host() + host.create_offer() + + async def _drive(): + FakePeerConnection.instances[0].fire("track", Track("video")) + assert host._viewer_video_task is not None + host._viewer_video_task.cancel() + + asyncio.run(_drive()) + + +def test_an_inbound_audio_track_is_ignored_unless_it_was_advertised(bridge): + host = _host() + host.create_offer() + FakePeerConnection.instances[0].fire("track", Track("audio")) + assert host._opus_audio_receiver is None + + +def test_an_advertised_audio_track_starts_an_opus_receiver(bridge, + monkeypatch): + receivers = [] + + class _Receiver: + def __init__(self) -> None: + self.consumed = [] + receivers.append(self) + + def consume(self, track): + self.consumed.append(track) + + monkeypatch.setattr( + "je_auto_control.utils.remote_desktop.webrtc_audio.OpusMicReceiver", + _Receiver, + ) + host = _host(config=WebRTCConfig(accept_viewer_audio_opus=True)) + host.create_offer() + track = Track("audio") + FakePeerConnection.instances[0].fire("track", track) + assert receivers[0].consumed == [track] + + +def test_a_second_audio_track_does_not_open_a_second_receiver(bridge, + monkeypatch): + class _Receiver: + def __init__(self) -> None: + self.consumed = [] + + def consume(self, track): + self.consumed.append(track) + + monkeypatch.setattr( + "je_auto_control.utils.remote_desktop.webrtc_audio.OpusMicReceiver", + _Receiver, + ) + host = _host(config=WebRTCConfig(accept_viewer_audio_opus=True)) + host.create_offer() + pc = FakePeerConnection.instances[0] + pc.fire("track", Track("audio")) + first = host._opus_audio_receiver + pc.fire("track", Track("audio")) + assert host._opus_audio_receiver is first + + +def test_a_missing_speaker_does_not_break_the_session(bridge, monkeypatch): + def _no_device(): + raise OSError("no output device") + + monkeypatch.setattr( + "je_auto_control.utils.remote_desktop.webrtc_audio.OpusMicReceiver", + _no_device, + ) + host = _host(config=WebRTCConfig(accept_viewer_audio_opus=True)) + host.create_offer() + FakePeerConnection.instances[0].fire("track", Track("audio")) + assert host._opus_audio_receiver is None + + +def test_viewer_frames_are_dropped_until_the_viewer_authenticates(): + # The video slot opens with the PeerConnection, which is before the + # token has been checked: frames arriving in that window are a stream + # from a peer we have not accepted yet. + host = _host() + seen = [] + host.set_viewer_video_callback(seen.append) + asyncio.run(host._consume_viewer_video(FrameTrack("f1", "f2"))) + assert seen == [] + + +def test_viewer_frames_reach_the_callback_once_authenticated(): + host = _host() + seen = [] + host.set_viewer_video_callback(seen.append) + host._authenticated = True + asyncio.run(host._consume_viewer_video(FrameTrack("f1", "f2"))) + assert seen == ["f1", "f2"] + + +def test_viewer_frames_with_no_listener_are_drained_and_discarded(): + # Nothing is registered until the GUI opens the viewer-screen window; + # the frames still have to be pulled or the receiver backs up. + host = _host() + host._authenticated = True + track = FrameTrack("f1", "f2") + asyncio.run(host._consume_viewer_video(track)) + assert track.frames == [] + + +def test_a_track_that_is_neither_audio_nor_video_is_ignored(bridge): + host = _host() + host.create_offer() + FakePeerConnection.instances[0].fire("track", Track("application")) + assert host._viewer_video_task is None + assert host._opus_audio_receiver is None + + +def test_a_raising_frame_callback_does_not_end_the_stream(): + host = _host() + host._authenticated = True + seen = [] + + def _cb(frame): + seen.append(frame) + raise RuntimeError("paint failed") + + host.set_viewer_video_callback(_cb) + asyncio.run(host._consume_viewer_video(FrameTrack("f1", "f2"))) + assert seen == ["f1", "f2"], "the second frame was still delivered" + + +def test_the_consume_task_clears_itself_when_the_stream_ends(): + host = _host() + host._viewer_video_task = "placeholder" + asyncio.run(host._consume_viewer_video(FrameTrack(ending=OSError("gone")))) + assert host._viewer_video_task is None + + +# --- teardown ----------------------------------------------------------------- + +def test_stopping_a_host_that_never_connected_is_a_no_op(bridge): + _host().stop() + assert not bridge.deferred + + +def test_stop_closes_the_peer_connection_and_forgets_the_channels(bridge): + host = _host() + host.create_offer() + host._authenticated = True + host.stop() + assert FakePeerConnection.instances[0].closed + assert host._pc is None + assert host._control_channel is None + assert host._files_channel is None + assert host.authenticated is False + assert host.connection_state == "closed" + + +def test_stop_stops_a_capture_the_host_created(): + host = _host() + track = Track("video") + host._video_track = track + asyncio.run(host._async_stop()) + assert track.stopped + + +def test_stop_leaves_a_relayed_track_alone(): + # `MultiViewerHost` owns the shared capture; stopping it from one + # session would blank the screen for every other viewer. + relayed = Track("video") + host = _host(external_video_track=relayed) + host._video_track = relayed + asyncio.run(host._async_stop()) + assert not relayed.stopped + + +def test_stop_releases_the_audio_and_mic_helpers(): + host = _host() + voice, receiver, mic = Stoppable(), Stoppable(), Stoppable() + host._host_voice_track = voice + host._opus_audio_receiver = receiver + host._mic_receiver = mic + asyncio.run(host._async_stop()) + assert (voice.stopped, receiver.stopped, mic.stopped) == (True, True, True) + assert host._host_voice_track is None + assert host._opus_audio_receiver is None + assert host._mic_receiver is None + + +def test_stop_cancels_the_viewer_video_task_and_the_auth_deadline(): + host = _host() + + async def _drive(): + task = asyncio.ensure_future(asyncio.sleep(10)) + handle = asyncio.get_event_loop().call_later(10, lambda: None) + host._viewer_video_task = task + host._auth_deadline_handle = handle + await host._async_stop() + return task, handle + + task, handle = asyncio.run(_drive()) + assert task.cancelled() + assert handle.cancelled() + assert host._auth_deadline_handle is None + + +def test_a_teardown_failure_does_not_abort_the_rest_of_the_teardown(): + host = _host() + voice = Stoppable(error=OSError("device gone")) + receiver = Stoppable() + host._host_voice_track = voice + host._opus_audio_receiver = receiver + asyncio.run(host._async_stop()) + assert receiver.stopped, "teardown continued past the failure" + + +def test_stop_quietly_ignores_a_collaborator_that_was_never_built(): + WebRTCDesktopHost._stop_quietly(None, "nothing") + + +def test_a_stop_that_times_out_is_reported_rather_than_raised(monkeypatch): + monkeypatch.setattr(host_module, "get_bridge", HangingBridge) + host = _host() + host._pc = object() + host.stop() # the GUI's close button must not raise diff --git a/test/unit_test/headless/test_webrtc_opus_audio.py b/test/unit_test/headless/test_webrtc_opus_audio.py new file mode 100644 index 00000000..0425c996 --- /dev/null +++ b/test/unit_test/headless/test_webrtc_opus_audio.py @@ -0,0 +1,395 @@ +"""The Opus mic uplink: a sounddevice thread on one end, asyncio on the other. + +`webrtc_audio` sits on a thread boundary in both directions and had no test +at all -- it needs aiortc *and* av to import, so it read 0% on every CI +square until the `[webrtc]` extra joined the measured install. + +What the tests below are actually about: + +* **The capture callback runs on the sounddevice thread.** It may not touch + the asyncio queue directly, so it hops through `call_soon_threadsafe` -- + and when the loop has already closed under it (the viewer disconnected + while a block was in flight) the `RuntimeError` that raises is the normal + case, not an error to report. +* **The queue drops the oldest block, not the newest.** A bounded queue is + what keeps mic latency from growing without limit when the encoder falls + behind; dropping the *newest* block would bound the queue just as well + and make the audio permanently stale. +* **`recv` mints the presentation timestamps itself.** aiortc packetises + what it is handed, so a pts that does not advance by exactly the sample + count is a stream that drifts against its own clock. +* **The receiver's drain loop is a containment boundary.** It runs as a + fire-and-forget task on the shared bridge loop; every way a track can end + -- cancellation, `MediaStreamError`, a decode failure, the player being + stopped underneath it -- has to end the loop quietly rather than take the + loop down with it. + +`AudioCapture` and `AudioPlayer` are replaced with recorders: they are the +sounddevice boundary, and sounddevice is not installed on a CI runner. The +frames are real `av.AudioFrame`s, because their layout is the contract +between this module and aiortc. +""" +from __future__ import annotations + +import asyncio + +import numpy as np +import pytest + +from headless._webrtc_doubles import FrameTrack +from je_auto_control.utils.remote_desktop import webrtc_audio as audio_mod +from je_auto_control.utils.remote_desktop.audio import AudioBackendError +from je_auto_control.utils.remote_desktop.webrtc_audio import ( + OpusMicAudioTrack, OpusMicReceiver, +) + + +class _FakeCapture: + instances = [] + + def __init__(self, *, on_block, device, sample_rate, channels, + block_frames) -> None: + self.on_block = on_block + self.device = device + self.sample_rate = sample_rate + self.channels = channels + self.block_frames = block_frames + self.started = False + self.stopped = False + self.stop_error = None + _FakeCapture.instances.append(self) + + def start(self) -> None: + self.started = True + + def stop(self) -> None: + if self.stop_error is not None: + raise self.stop_error + self.stopped = True + + +class _FakePlayer: + instances = [] + + def __init__(self, *, device, sample_rate, channels) -> None: + self.device = device + self.sample_rate = sample_rate + self.channels = channels + self.is_running = False + self.played = [] + self.stopped = False + self.stop_error = None + _FakePlayer.instances.append(self) + + def start(self) -> None: + self.is_running = True + + def play(self, pcm_bytes) -> None: + self.played.append(pcm_bytes) + + def stop(self) -> None: + if self.stop_error is not None: + raise self.stop_error + self.stopped = True + + +class _Frame: + """Stands in for an ``av.AudioFrame`` the decoder handed us.""" + + def __init__(self, array=None, error=None) -> None: + self._array = array + self._error = error + + def to_ndarray(self): + if self._error is not None: + raise self._error + return self._array + + +@pytest.fixture(autouse=True) +def fake_audio_backend(monkeypatch): + """Swap the sounddevice boundary out; keep everything above it real.""" + _FakeCapture.instances = [] + _FakePlayer.instances = [] + monkeypatch.setattr(audio_mod, "is_audio_backend_available", lambda: True) + monkeypatch.setattr(audio_mod, "AudioCapture", _FakeCapture) + monkeypatch.setattr(audio_mod, "AudioPlayer", _FakePlayer) + yield + _FakeCapture.instances = [] + _FakePlayer.instances = [] + + +# --- the viewer's outbound track ---------------------------------------------- + +def test_track_refuses_to_start_without_a_sounddevice_backend(monkeypatch): + monkeypatch.setattr(audio_mod, "is_audio_backend_available", lambda: False) + + async def _build(): + OpusMicAudioTrack() + + with pytest.raises(AudioBackendError, match="sounddevice"): + asyncio.run(_build()) + + +def test_track_starts_capture_at_the_rate_opus_wants(): + async def _build(): + return OpusMicAudioTrack() + + track = asyncio.run(_build()) + [capture] = _FakeCapture.instances + assert capture.started + assert (capture.sample_rate, capture.channels) == (48000, 1) + assert capture.block_frames == 960, "20 ms at 48 kHz" + assert track.kind == "audio" + + +def test_track_passes_the_chosen_input_device_through(): + async def _build(): + return OpusMicAudioTrack(sample_rate=16000, channels=2, device=3) + + asyncio.run(_build()) + [capture] = _FakeCapture.instances + assert (capture.device, capture.sample_rate, capture.channels) == ( + 3, 16000, 2, + ) + + +def test_a_second_start_does_not_open_a_second_stream(): + async def _build(): + track = OpusMicAudioTrack() + track._start_capture() + return track + + asyncio.run(_build()) + assert len(_FakeCapture.instances) == 1 + + +def test_a_captured_block_reaches_the_queue_from_the_capture_thread(): + async def _drive(): + track = OpusMicAudioTrack() + capture = _FakeCapture.instances[0] + capture.on_block(b"\x01\x02") + await asyncio.sleep(0) # let call_soon_threadsafe land + return track._queue.get_nowait() + + assert asyncio.run(_drive()) == b"\x01\x02" + + +def test_a_block_arriving_after_the_loop_closed_is_dropped_silently(): + # The sounddevice thread outlives the loop by design: it is stopped from + # `stop()`, which itself runs after the viewer has already gone away. + async def _build(): + return OpusMicAudioTrack() + + track = asyncio.run(_build()) + track._on_block(b"\x00\x00") # the loop from _build is closed now + + +def test_a_full_queue_drops_the_oldest_block(): + async def _drive(): + track = OpusMicAudioTrack() + for index in range(track._queue.maxsize): + track._enqueue(bytes([index])) + track._enqueue(b"\xff") + first = track._queue.get_nowait() + drained = [first] + while not track._queue.empty(): + drained.append(track._queue.get_nowait()) + return drained + + drained = asyncio.run(_drive()) + assert drained[0] == b"\x01", "the oldest block made room" + assert drained[-1] == b"\xff", "the newest one is kept" + + +def test_recv_turns_pcm_into_an_interleaved_s16_frame(): + pcm = np.arange(960, dtype=np.int16).tobytes() + + async def _drive(): + track = OpusMicAudioTrack() + track._enqueue(pcm) + return await track.recv() + + frame = asyncio.run(_drive()) + assert frame.sample_rate == 48000 + assert frame.samples == 960 + assert frame.time_base.denominator == 48000 + assert frame.pts == 0 + + +def test_recv_advances_the_timestamp_by_the_samples_it_sent(): + pcm = np.zeros(960, dtype=np.int16).tobytes() + + async def _drive(): + track = OpusMicAudioTrack() + track._enqueue(pcm) + track._enqueue(pcm) + first = await track.recv() + second = await track.recv() + return first.pts, second.pts + + # A pts that does not advance by exactly one block's worth of samples is + # a stream that drifts against the clock aiortc packetises it with. + assert asyncio.run(_drive()) == (0, 960) + + +def test_recv_counts_samples_per_channel_on_a_stereo_capture(): + pcm = np.zeros(960 * 2, dtype=np.int16).tobytes() + + async def _drive(): + track = OpusMicAudioTrack(channels=2) + track._enqueue(pcm) + track._enqueue(pcm) + await track.recv() + return (await track.recv()).pts + + assert asyncio.run(_drive()) == 960 + + +def test_stopping_the_track_stops_the_capture_once(): + async def _drive(): + track = OpusMicAudioTrack() + track.stop() + track.stop() + return track + + asyncio.run(_drive()) + [capture] = _FakeCapture.instances + assert capture.stopped + + +def test_a_capture_that_fails_to_close_still_leaves_the_track_stopped(): + async def _drive(): + track = OpusMicAudioTrack() + _FakeCapture.instances[0].stop_error = OSError("device unplugged") + track.stop() + return track + + track = asyncio.run(_drive()) + assert track._capture is None + + +# --- the host's inbound receiver ---------------------------------------------- + +def test_receiver_refuses_to_start_without_a_sounddevice_backend(monkeypatch): + monkeypatch.setattr(audio_mod, "is_audio_backend_available", lambda: False) + with pytest.raises(AudioBackendError, match="sounddevice"): + OpusMicReceiver() + + +def test_receiver_opens_the_player_at_the_requested_rate(): + OpusMicReceiver(sample_rate=16000, channels=2, device=4) + [player] = _FakePlayer.instances + assert (player.device, player.sample_rate, player.channels) == ( + 4, 16000, 2, + ) + assert player.is_running + + +def test_receiver_plays_the_decoded_frames_it_drains(): + samples = np.array([[1, 2, 3, 4]], dtype=np.int16) + + async def _drive(): + receiver = OpusMicReceiver() + track = FrameTrack(_Frame(samples)) + receiver.consume(track) + await receiver._task + return receiver + + asyncio.run(_drive()) + [player] = _FakePlayer.instances + assert player.played == [samples.tobytes()] + + +def test_receiver_converts_a_float_frame_before_playing_it(): + # av hands back whatever the decoder produced; the player only speaks + # int16 PCM, so a float layout has to be narrowed rather than passed on. + async def _drive(): + receiver = OpusMicReceiver() + receiver.consume(FrameTrack(_Frame(np.array([[1.0, 2.0]], + dtype=np.float32)))) + await receiver._task + + asyncio.run(_drive()) + [player] = _FakePlayer.instances + assert player.played == [np.array([[1, 2]], dtype=np.int16).tobytes()] + + +def test_a_second_consume_does_not_start_a_second_drain(): + async def _drive(): + receiver = OpusMicReceiver() + track = FrameTrack(_Frame(np.array([[1]], dtype=np.int16))) + receiver.consume(track) + first = receiver._task + receiver.consume(FrameTrack()) + assert receiver._task is first + await first + + asyncio.run(_drive()) + + +def test_an_undecodable_frame_is_skipped_rather_than_ending_the_stream(): + good = np.array([[5, 6]], dtype=np.int16) + + async def _drive(): + receiver = OpusMicReceiver() + receiver.consume(FrameTrack(_Frame(error=ValueError("bad plane")), + _Frame(good))) + await receiver._task + + asyncio.run(_drive()) + [player] = _FakePlayer.instances + assert player.played == [good.tobytes()], "the good frame still played" + + +def test_the_drain_stops_when_the_player_is_no_longer_running(): + async def _drive(): + receiver = OpusMicReceiver() + _FakePlayer.instances[0].is_running = False + track = FrameTrack(_Frame(np.array([[1]], dtype=np.int16)), + _Frame(np.array([[2]], dtype=np.int16))) + receiver.consume(track) + await receiver._task + return track + + track = asyncio.run(_drive()) + assert len(track.frames) == 1, "it did not keep pulling from the track" + assert _FakePlayer.instances[0].played == [] + + +def test_the_drain_ends_quietly_when_the_track_dies(): + # The viewer closing its tab surfaces here as an OSError out of recv; + # this task runs on the shared bridge loop, so it may not propagate. + async def _drive(): + receiver = OpusMicReceiver() + receiver.consume(FrameTrack(ending=OSError("connection reset"))) + await receiver._task + + asyncio.run(_drive()) + + +def test_stopping_the_receiver_cancels_the_drain_and_closes_the_player(): + async def _drive(): + receiver = OpusMicReceiver() + receiver.consume(FrameTrack(_Frame(np.array([[1]], dtype=np.int16)))) + task = receiver._task + receiver.stop() + assert receiver._task is None + await asyncio.sleep(0) + return task + + task = asyncio.run(_drive()) + assert task.cancelled() or task.done() + assert _FakePlayer.instances[0].stopped + + +def test_stopping_a_receiver_that_never_consumed_still_closes_the_player(): + receiver = OpusMicReceiver() + receiver.stop() + assert _FakePlayer.instances[0].stopped + + +def test_a_player_that_fails_to_close_does_not_escape_stop(): + receiver = OpusMicReceiver() + _FakePlayer.instances[0].stop_error = OSError("stream already closed") + receiver.stop() diff --git a/test/unit_test/headless/test_webrtc_stats_poller.py b/test/unit_test/headless/test_webrtc_stats_poller.py new file mode 100644 index 00000000..a4c27423 --- /dev/null +++ b/test/unit_test/headless/test_webrtc_stats_poller.py @@ -0,0 +1,240 @@ +"""The arithmetic that decides the remote-desktop link is going bad. + +`StatsPoller` turns aiortc's cumulative `getStats()` counters into the per-sample +rates `AdaptiveBitrateController` acts on: bitrate, fps, packet loss, RTT and +jitter. Every one of those is a *delta* between two samples, which is where this +kind of code goes wrong -- a first sample with no predecessor, a counter that +resets when a track is replaced, a report that names no candidate pair. Get one +wrong and the controller drops the frame rate on a link that is fine, or holds a +high rate on one that is not. + +Nothing imported this module on any CI square before the `[webrtc]` extra joined +the measured install: it is reachable only through `webrtc_transport`, which +raises ImportError at module level without aiortc. + +The poller is driven here through `_sample()` on a bare event loop rather than +through `start()`, because `start()` submits to the shared asyncio bridge and +what is under test is the arithmetic, not the bridge. +""" +import asyncio + +import pytest + +from je_auto_control.utils.remote_desktop import webrtc_stats +from je_auto_control.utils.remote_desktop.webrtc_stats import ( + StatsPoller, StatsSnapshot, +) + + +class _Entry: + """One `RTCStats` row: aiortc exposes these as attributes.""" + + def __init__(self, **fields): + self.__dict__.update(fields) + + +class _PeerConnection: + """Hands back one prepared report per `getStats()` call.""" + + def __init__(self, *reports): + self._reports = list(reports) + self.calls = 0 + + async def getStats(self): # noqa: N802 # reason: the aiortc name + self.calls += 1 + report = self._reports.pop(0) if self._reports else {} + return {str(index): entry for index, entry in enumerate(report)} + + +def _inbound(**fields): + return _Entry(type="inbound-rtp", kind="video", **fields) + + +class _Clock: + """A monotonic clock the test drives. + + Replaces the module's own ``time`` binding rather than ``time.monotonic`` + itself: asyncio reads the real clock while these coroutines run, and a + fake installed globally makes the event loop read the fake too. + """ + + def __init__(self, *ticks): + self._ticks = list(ticks) + + def monotonic(self): + return self._ticks.pop(0) if len(self._ticks) > 1 else self._ticks[0] + + +def _pin_clock(monkeypatch, *ticks): + monkeypatch.setattr(webrtc_stats, "time", _Clock(*ticks)) + + +def _sample(poller): + """Take one sample.""" + return asyncio.run(poller._sample()) + + +# === A poller with nothing to poll ========================================== + +def test_a_poller_without_a_peer_connection_samples_nothing(): + assert asyncio.run(StatsPoller(None, lambda snap: None)._sample()) is None + + +def test_an_empty_report_yields_an_empty_snapshot(): + poller = StatsPoller(_PeerConnection([]), lambda snap: None) + snapshot = _sample(poller) + assert snapshot.to_dict() == StatsSnapshot().to_dict() + + +def test_the_poll_interval_has_a_floor(): + """A caller asking for a 1 ms poll would spin the loop, not measure it.""" + assert StatsPoller(None, lambda snap: None, interval_s=0.001)._interval == 0.25 + assert StatsPoller(None, lambda snap: None, interval_s=5)._interval == 5.0 + + +# === Rates need two samples ================================================= + +def test_the_first_sample_reports_no_rate(): + """There is nothing to subtract from yet; a rate would be invented.""" + poller = StatsPoller( + _PeerConnection([_inbound(bytesReceived=10_000, framesDecoded=30)]), + lambda snap: None) + snapshot = _sample(poller) + assert snapshot.bitrate_kbps is None + assert snapshot.fps is None + + +def test_the_second_sample_reports_the_rate_between_them(monkeypatch): + reports = [[_inbound(bytesReceived=0, framesDecoded=0)], + [_inbound(bytesReceived=125_000, framesDecoded=60)]] + poller = StatsPoller(_PeerConnection(*reports), lambda snap: None) + _pin_clock(monkeypatch, 100.0, 102.0) + _sample(poller) + snapshot = _sample(poller) + # 125,000 bytes over 2 s = 500 kbit/s; 60 frames over 2 s = 30 fps. + assert snapshot.bitrate_kbps == pytest.approx(500.0) + assert snapshot.fps == pytest.approx(30.0) + + +def test_a_counter_that_went_backwards_reports_no_rate(monkeypatch): + """A replaced track restarts the counters; a negative delta is not a rate.""" + reports = [[_inbound(bytesReceived=125_000, framesDecoded=60)], + [_inbound(bytesReceived=10, framesDecoded=1)]] + poller = StatsPoller(_PeerConnection(*reports), lambda snap: None) + _pin_clock(monkeypatch, 100.0, 102.0) + _sample(poller) + snapshot = _sample(poller) + assert snapshot.bitrate_kbps is None + assert snapshot.fps is None + + +def test_two_samples_at_the_same_instant_report_no_rate(monkeypatch): + """Dividing by a zero interval is the other way to invent a number.""" + reports = [[_inbound(bytesReceived=0)], [_inbound(bytesReceived=125_000)]] + poller = StatsPoller(_PeerConnection(*reports), lambda snap: None) + _pin_clock(monkeypatch, 100.0) + _sample(poller) + assert _sample(poller).bitrate_kbps is None + + +# === Packet loss is measured over the interval, not since the connection ==== + +def test_loss_is_the_recent_ratio_not_the_lifetime_one(monkeypatch): + """A link that lost 50 packets an hour ago is not a link losing them now.""" + reports = [[_inbound(packetsReceived=1_000, packetsLost=50)], + [_inbound(packetsReceived=1_100, packetsLost=50)]] + poller = StatsPoller(_PeerConnection(*reports), lambda snap: None) + _pin_clock(monkeypatch, 100.0, 101.0) + _sample(poller) + assert _sample(poller).packet_loss_pct == pytest.approx(0.0) + + +def test_loss_in_the_last_interval_is_reported(monkeypatch): + reports = [[_inbound(packetsReceived=1_000, packetsLost=0)], + [_inbound(packetsReceived=1_090, packetsLost=10)]] + poller = StatsPoller(_PeerConnection(*reports), lambda snap: None) + _pin_clock(monkeypatch, 100.0, 101.0) + _sample(poller) + # 10 lost out of 100 sent in the interval. + assert _sample(poller).packet_loss_pct == pytest.approx(10.0) + + +def test_a_report_with_no_packets_at_all_reports_no_loss(): + poller = StatsPoller( + _PeerConnection([_inbound(packetsReceived=0, packetsLost=0)]), + lambda snap: None) + assert _sample(poller).packet_loss_pct is None + + +def test_missing_counters_read_as_zero_rather_than_raising(): + """aiortc omits attributes it has no value for.""" + poller = StatsPoller(_PeerConnection([_inbound()]), lambda snap: None) + assert _sample(poller).packet_loss_pct is None + + +def test_a_null_counter_reads_as_zero(): + poller = StatsPoller( + _PeerConnection([_inbound(packetsReceived=None, packetsLost=None)]), + lambda snap: None) + assert _sample(poller).packet_loss_pct is None + + +# === Where RTT and jitter come from ========================================= + +def test_the_remote_report_supplies_rtt_and_jitter_in_milliseconds(): + entry = _Entry(type="remote-inbound-rtp", roundTripTime=0.125, jitter=0.004) + poller = StatsPoller(_PeerConnection([entry]), lambda snap: None) + snapshot = _sample(poller) + assert snapshot.rtt_ms == pytest.approx(125.0) + assert snapshot.jitter_ms == pytest.approx(4.0) + + +def test_the_candidate_pair_supplies_rtt_when_the_remote_report_does_not(): + entry = _Entry(type="candidate-pair", currentRoundTripTime=0.05) + poller = StatsPoller(_PeerConnection([entry]), lambda snap: None) + assert _sample(poller).rtt_ms == pytest.approx(50.0) + + +def test_the_remote_report_wins_over_the_candidate_pair(): + """Both are in the report; the one measured end to end is the real one.""" + report = [_Entry(type="remote-inbound-rtp", roundTripTime=0.125), + _Entry(type="candidate-pair", currentRoundTripTime=0.05)] + poller = StatsPoller(_PeerConnection(report), lambda snap: None) + assert _sample(poller).rtt_ms == pytest.approx(125.0) + + +def test_a_candidate_pair_with_no_measurement_is_ignored(): + entry = _Entry(type="candidate-pair", currentRoundTripTime=None) + poller = StatsPoller(_PeerConnection([entry]), lambda snap: None) + assert _sample(poller).rtt_ms is None + + +def test_audio_inbound_rows_do_not_drive_the_video_rate(monkeypatch): + """`kind` is what separates the two; only video feeds the controller.""" + audio = _Entry(type="inbound-rtp", kind="audio", + bytesReceived=1_000_000, framesDecoded=0) + poller = StatsPoller(_PeerConnection([audio], [audio]), lambda snap: None) + _pin_clock(monkeypatch, 100.0, 101.0) + _sample(poller) + assert _sample(poller).bitrate_kbps is None + + +def test_an_unknown_stat_type_is_ignored(): + poller = StatsPoller(_PeerConnection([_Entry(type="transport")]), + lambda snap: None) + assert _sample(poller).to_dict() == StatsSnapshot().to_dict() + + +def test_a_row_with_no_type_at_all_is_ignored(): + poller = StatsPoller(_PeerConnection([_Entry(bytesReceived=5)]), + lambda snap: None) + assert _sample(poller).to_dict() == StatsSnapshot().to_dict() + + +# === Stopping ================================================================ + +def test_stopping_a_poller_that_never_started_is_harmless(): + poller = StatsPoller(_PeerConnection(), lambda snap: None) + poller.stop() + assert poller._stopped is True + assert poller._task is None diff --git a/test/unit_test/headless/test_webrtc_transport.py b/test/unit_test/headless/test_webrtc_transport.py new file mode 100644 index 00000000..716185d1 --- /dev/null +++ b/test/unit_test/headless/test_webrtc_transport.py @@ -0,0 +1,424 @@ +"""The plumbing both WebRTC ends stand on: bridge, capture, cursor, ICE. + +`webrtc_transport` is the module every other WebRTC module imports, and the +one that raises ImportError at module level without aiortc -- so before the +`[webrtc]` extra joined the measured install, nothing here ran on any CI +square at all. + +Four things in it are worth pinning down, and none of them need a peer: + +* **The bridge is a singleton around one background loop.** `get_bridge()` + hands the same object to host, viewer and every GUI panel, so `start()` + has to be idempotent under a lock -- a second loop would silently split + the DataChannel sends from the PeerConnection they belong to. +* **`_draw_cursor_overlay` writes into a slice of the captured frame.** The + slice is clamped at the array edges but the circle arithmetic is not, so + the ring is computed in absolute coordinates and applied to a window that + may start anywhere. A cursor at the very corner is the case that would + raise if the two disagreed. +* **`_resolve_monitor` is fed an index that comes from the GUI**, where + monitors are listed per mss numbering (1-based, 0 = "all"). An index the + user picked before unplugging a screen must land somewhere real rather + than raise out of the capture thread. +* **`wait_for_ice_gathering` deliberately gives up.** Its timeout branch is + the one that ships a half-gathered SDP rather than hanging the offer, and + it is only reachable when nothing ever completes. + +The capture path is exercised against a fake grabber: `_capture_frame` +caches one mss instance per thread in a module-level `threading.local()`, +which is worth a test of its own because a leak there is one screen grabber +per capture rather than one per thread. +""" +from __future__ import annotations + +import asyncio +import threading +import time + +import numpy as np +import pytest + +from headless._webrtc_doubles import FakePeerConnection +from je_auto_control.utils.remote_desktop import webrtc_transport as transport +from je_auto_control.utils.remote_desktop.webrtc_transport import ( + BANDWIDTH_PRESETS, ScreenVideoTrack, WebRTCConfig, _AsyncioBridge, + _capture_frame, _draw_cursor_overlay, _resolve_monitor, fps_for_preset, + get_bridge, wait_for_ice_gathering, +) + + +# --- fakes -------------------------------------------------------------------- + +class _Grab: + """What ``mss.grab()`` returns: a BGRA buffer plus its dimensions.""" + + def __init__(self, width: int, height: int, fill: int = 7) -> None: + self.width = width + self.height = height + self.bgra = bytes([fill, fill + 1, fill + 2, 255]) * (width * height) + + +class _FakeSct: + def __init__(self, monitors=None) -> None: + self.monitors = monitors if monitors is not None else [ + {"left": 0, "top": 0, "width": 8, "height": 4}, + {"left": 0, "top": 0, "width": 4, "height": 2}, + ] + self.grabs = [] + + def grab(self, monitor): + self.grabs.append(monitor) + return _Grab(monitor["width"], monitor["height"]) + + +@pytest.fixture +def fake_grabber(monkeypatch): + """Install a fake mss grabber and clear the per-thread capture cache.""" + sct = _FakeSct() + calls = [] + + def _grabber(): + calls.append(1) + return sct + + monkeypatch.setattr(transport, "mss_grabber", _grabber) + if hasattr(transport._capture_local, "sct"): + del transport._capture_local.sct + yield sct, calls + if hasattr(transport._capture_local, "sct"): + del transport._capture_local.sct + + +# --- presets and config ------------------------------------------------------- + +def test_fps_for_preset_is_case_insensitive_and_falls_back_to_auto(): + assert fps_for_preset("LOW") == BANDWIDTH_PRESETS["low"]["fps"] + assert fps_for_preset("high") == 30 + # The GUI persists the preset name; a config written by a newer build + # must not make an older one raise out of the capture setup. + assert fps_for_preset("ludicrous") == BANDWIDTH_PRESETS["auto"]["fps"] + + +def test_config_turns_stun_list_into_ice_servers(): + config = WebRTCConfig(ice_servers=["stun:a:1", "stun:b:2"]) + rtc = config.to_rtc_configuration() + assert [server.urls for server in rtc.iceServers] == ["stun:a:1", "stun:b:2"] + + +def test_config_appends_turn_server_with_its_credentials(): + config = WebRTCConfig( + ice_servers=["stun:a:1"], turn_url="turn:relay:3478", + turn_username="user", turn_credential="secret", + ) + turn = config.to_rtc_configuration().iceServers[-1] + assert (turn.urls, turn.username, turn.credential) == ( + "turn:relay:3478", "user", "secret", + ) + + +def test_config_without_turn_url_adds_no_extra_server(): + config = WebRTCConfig(ice_servers=["stun:a:1"], turn_username="user") + # Credentials with no URL are half-filled GUI state, not a server. + assert len(config.to_rtc_configuration().iceServers) == 1 + + +# --- the shared asyncio bridge ------------------------------------------------ + +def test_bridge_start_is_idempotent_and_runs_submitted_coroutines(): + bridge = _AsyncioBridge() + try: + loop = bridge.start() + assert bridge.start() is loop, "a second loop would split the session" + assert loop.is_running() + + async def _answer(): + return 42 + + assert bridge.submit(_answer()).result(timeout=5.0) == 42 + finally: + bridge.stop() + + +def test_bridge_call_soon_runs_the_callable_on_the_loop_thread(): + bridge = _AsyncioBridge() + seen = {} + done = threading.Event() + + def _record(value): + seen["thread"] = threading.current_thread().name + seen["value"] = value + done.set() + + try: + bridge.call_soon(_record, "payload") + assert done.wait(timeout=5.0) + finally: + bridge.stop() + assert seen["value"] == "payload" + assert seen["thread"] == "webrtc-loop" + + +def test_bridge_stop_is_safe_before_start_and_after_stop(): + bridge = _AsyncioBridge() + bridge.stop() # never started: nothing to join + bridge.start() + bridge.stop() + bridge.stop() # already torn down + + +def test_get_bridge_returns_the_process_wide_instance(): + assert get_bridge() is get_bridge() + + +# --- cursor overlay ----------------------------------------------------------- + +def _white(height=48, width=48): + return np.full((height, width, 3), 255, dtype=np.uint8) + + +def test_cursor_overlay_draws_a_yellow_ring_around_a_black_core(): + frame = _white() + _draw_cursor_overlay(frame, 24, 24) + assert tuple(frame[24, 24]) == (0, 0, 0), "core" + assert tuple(frame[24, 24 + 8]) == (0, 255, 255), "ring at radius 8" + assert tuple(frame[0, 0]) == (255, 255, 255), "far corner untouched" + + +@pytest.mark.parametrize("x,y", [(-1, 10), (10, -1), (48, 10), (10, 48)]) +def test_cursor_overlay_ignores_positions_outside_the_frame(x, y): + frame = _white() + _draw_cursor_overlay(frame, x, y) + assert (frame == 255).all(), "an off-screen cursor must not paint" + + +def test_cursor_overlay_clamps_its_window_at_the_frame_corner(): + # The ring is computed in absolute coordinates and written into a slice + # that stops at the edge; at (0, 0) three quarters of it are off-frame. + frame = _white() + _draw_cursor_overlay(frame, 0, 0) + assert tuple(frame[0, 0]) == (0, 0, 0) + assert tuple(frame[0, 8]) == (0, 255, 255) + + +# --- monitor resolution and capture ------------------------------------------- + +def test_resolve_monitor_returns_the_requested_entry(): + sct = _FakeSct() + assert _resolve_monitor(sct, 1) is sct.monitors[1] + + +@pytest.mark.parametrize("index", [-1, 2, 99]) +def test_resolve_monitor_falls_back_to_the_first_screen(index): + # A stale index (the user unplugged the screen they had picked) must + # land on a real monitor rather than raise inside the capture thread. + sct = _FakeSct() + assert _resolve_monitor(sct, index) is sct.monitors[1] + + +def test_resolve_monitor_falls_back_to_the_only_entry_when_alone(): + sct = _FakeSct(monitors=[{"left": 0, "top": 0, "width": 2, "height": 2}]) + assert _resolve_monitor(sct, 5) is sct.monitors[0] + + +def test_resolve_monitor_raises_when_mss_reports_nothing(): + with pytest.raises(RuntimeError, match="no monitors"): + _resolve_monitor(_FakeSct(monitors=[]), 1) + + +def test_capture_frame_drops_alpha_and_returns_a_contiguous_bgr_array( + fake_grabber): + monitor = {"left": 0, "top": 0, "width": 4, "height": 2} + arr = _capture_frame(monitor) + assert arr.shape == (2, 4, 3) + assert arr.dtype == np.uint8 + # av.VideoFrame.from_ndarray requires contiguity; the BGRA slice is not. + assert arr.flags["C_CONTIGUOUS"] + assert tuple(arr[0, 0]) == (7, 8, 9), "the alpha byte is dropped, not read" + + +def test_capture_frame_reuses_one_grabber_per_thread(fake_grabber): + sct, calls = fake_grabber + monitor = {"left": 0, "top": 0, "width": 4, "height": 2} + _capture_frame(monitor) + _capture_frame(monitor) + assert len(calls) == 1, "one mss instance per thread, not per frame" + assert len(sct.grabs) == 2 + + +# --- ScreenVideoTrack --------------------------------------------------------- + +@pytest.fixture +def track_factory(): + """Build ScreenVideoTracks and shut their capture executors down.""" + made = [] + + def _make(**kwargs): + track = ScreenVideoTrack(**kwargs) + made.append(track) + return track + + yield _make + for track in made: + track.stop() + + +@pytest.mark.parametrize("requested,expected", [ + (0, 1), (-5, 1), (24, 24), (61, 60), (1000, 60), +]) +def test_track_clamps_its_frame_rate(track_factory, requested, expected): + assert track_factory(fps=requested).fps == expected + + +def test_set_target_fps_updates_the_period_it_sleeps_on(track_factory): + track = track_factory(fps=10) + track.set_target_fps(20) + assert track.fps == 20 + assert track._period == pytest.approx(0.05) + + +def test_set_target_fps_ignores_a_repeat_of_the_current_rate(track_factory): + track = track_factory(fps=24) + period = track._period + track.set_target_fps(24) + assert track._period is period + + +def test_set_target_fps_clamps_the_adaptive_controller_too(track_factory): + # The bandwidth controller feeds this from observed RTT, so the clamp + # is the only thing between a bad measurement and a 1/0 period. + track = track_factory(fps=24) + track.set_target_fps(0) + assert track.fps == 1 + + +def test_track_resolves_a_region_without_asking_mss(track_factory, + fake_grabber): + _, calls = fake_grabber + track = track_factory(region=(10, 20, 30, 40)) + assert track._resolve() == {"left": 10, "top": 20, + "width": 30, "height": 40} + assert not calls, "a fixed region needs no monitor enumeration" + + +def test_track_resolves_and_caches_the_monitor(track_factory, fake_grabber): + sct, _ = fake_grabber + track = track_factory(monitor_index=1) + assert track._resolve() is sct.monitors[1] + assert track._resolve() is sct.monitors[1] + + +def test_set_target_monitor_invalidates_the_cached_lookup(track_factory, + fake_grabber): + sct, _ = fake_grabber + track = track_factory(monitor_index=1) + track._resolve() + track.set_target_monitor(0) + assert track._resolve() is sct.monitors[0] + + +def test_track_recv_returns_a_bgr_video_frame_with_a_timestamp( + track_factory, monkeypatch): + captured = np.full((4, 6, 3), 255, dtype=np.uint8) + monkeypatch.setattr(transport, "_capture_frame", lambda monitor: captured) + monkeypatch.setattr(transport, "_get_cursor_position", lambda: None) + track = track_factory(region=(0, 0, 6, 4), fps=60, show_cursor=True) + frame = asyncio.run(track.recv()) + assert (frame.width, frame.height) == (6, 4) + assert frame.time_base is not None + assert (frame.to_ndarray(format="bgr24") == 255).all() + + +def test_track_recv_overlays_the_cursor_in_monitor_local_coordinates( + track_factory, monkeypatch): + captured = np.full((48, 48, 3), 255, dtype=np.uint8) + monkeypatch.setattr(transport, "_capture_frame", lambda monitor: captured) + # Absolute (124, 224) on a monitor pinned at (100, 200) is local (24, 24). + monkeypatch.setattr(transport, "_get_cursor_position", lambda: (124, 224)) + track = track_factory(region=(100, 200, 48, 48), fps=60) + frame = asyncio.run(track.recv()) + arr = frame.to_ndarray(format="bgr24") + assert tuple(arr[24, 24]) == (0, 0, 0) + + +def test_track_recv_skips_the_overlay_when_the_cursor_is_unknown( + track_factory, monkeypatch): + captured = np.full((48, 48, 3), 255, dtype=np.uint8) + monkeypatch.setattr(transport, "_capture_frame", lambda monitor: captured) + # Wayland and headless X both leave the position unavailable. + monkeypatch.setattr(transport, "_get_cursor_position", lambda: None) + track = track_factory(region=(0, 0, 48, 48), fps=60, show_cursor=True) + frame = asyncio.run(track.recv()) + assert (frame.to_ndarray(format="bgr24") == 255).all() + + +def test_track_recv_paces_itself_at_the_target_rate(track_factory, + monkeypatch): + # Real time, not a patched `asyncio.sleep`: `transport.asyncio` IS the + # asyncio module, so patching through it also rewires aiortc's own + # `next_timestamp`, and the recorded delays stop being ours. 10 fps + # keeps the whole test inside a tenth of a second. + captured = np.full((4, 4, 3), 255, dtype=np.uint8) + monkeypatch.setattr(transport, "_capture_frame", lambda monitor: captured) + monkeypatch.setattr(transport, "_get_cursor_position", lambda: None) + track = track_factory(region=(0, 0, 4, 4), fps=10, show_cursor=False) + + started = time.monotonic() + asyncio.run(track.recv()) + first = time.monotonic() - started + asyncio.run(track.recv()) + second = time.monotonic() - started - first + + # The first frame goes out immediately; the second waits out the period. + assert first < 0.05 + assert second >= 0.05 + + +def test_track_stop_shuts_the_capture_executor_down(): + track = ScreenVideoTrack(region=(0, 0, 4, 4)) + track.stop() + with pytest.raises(RuntimeError): + track._executor.submit(lambda: None) + + +# --- ICE gathering ------------------------------------------------------------ + +def test_wait_for_ice_gathering_returns_at_once_when_already_complete(): + pc = FakePeerConnection() + pc.iceGatheringState = "complete" + asyncio.run(wait_for_ice_gathering(pc)) + assert not pc.handlers, "no need to subscribe to a finished gather" + + +def test_wait_for_ice_gathering_resolves_on_the_state_change(): + pc = FakePeerConnection() + + async def _drive(): + waiter = asyncio.ensure_future(wait_for_ice_gathering(pc, timeout=5.0)) + await asyncio.sleep(0) + pc.complete_ice_gathering() + await waiter + + asyncio.run(_drive()) + + +def test_wait_for_ice_gathering_ignores_intermediate_states(): + pc = FakePeerConnection() + + async def _drive(): + waiter = asyncio.ensure_future(wait_for_ice_gathering(pc, timeout=5.0)) + await asyncio.sleep(0) + pc.iceGatheringState = "gathering" + pc.handlers["icegatheringstatechange"]() + assert not waiter.done() + pc.complete_ice_gathering() + await waiter + + asyncio.run(_drive()) + + +def test_wait_for_ice_gathering_gives_up_and_sends_what_it_has(): + # The timeout is the branch that ships a half-gathered SDP instead of + # hanging create_offer() until its own 12 s future times out. + pc = FakePeerConnection() + asyncio.run(wait_for_ice_gathering(pc, timeout=0.05)) + assert pc.iceGatheringState == "new" diff --git a/test/unit_test/headless/test_webrtc_viewer_control.py b/test/unit_test/headless/test_webrtc_viewer_control.py new file mode 100644 index 00000000..c66d41d5 --- /dev/null +++ b/test/unit_test/headless/test_webrtc_viewer_control.py @@ -0,0 +1,528 @@ +"""What the viewer says to the host, and what it believes coming back. + +The session half is in `test_webrtc_viewer_session.py`. This file covers the +control channel: the request verbs the GUI drives, and the dispatch table +that decides what an inbound envelope means. + +The direction matters. On the host every inbound message is a boundary to be +refused; here every inbound message is a *claim by the host about this +session* -- that the token was accepted, that the session is read-only, that +the inbox holds these files -- and the viewer acts on it. So the tests below +are about the viewer believing exactly what it was told and no more: + +* **`auth_ok` is what flips the session live**, and it carries both the + read-only flag and the host's stable fingerprint in the same envelope. + A fingerprint that arrives empty must not overwrite one already shown to + the user, because that is the string they compared out-of-band. +* **`permissions` and `read_only` are two spellings of the same state.** + The newer envelope carries five flags; the viewer only mirrors one of them + (input), and a missing flag has to default to permitted, not to denied -- + the GUI greys out its own controls from this. +* **Every callback here is a Qt slot** reached from the asyncio thread, so + each one is wrapped: a raising GUI must not take the DataChannel down. + +`get_bridge` is replaced (from `headless._webrtc_doubles`) with one that +runs the callback inline; otherwise every `_send` would queue onto a loop no +test is running. +""" +from __future__ import annotations + +import asyncio +import json + +import pytest + +from headless._webrtc_doubles import Bridge, Channel, MicSender +from je_auto_control.utils.remote_desktop import webrtc_viewer as viewer_module +from je_auto_control.utils.remote_desktop.webrtc_viewer import ( + WebRTCDesktopViewer, +) + + +@pytest.fixture(autouse=True) +def bridge(monkeypatch): + fake = Bridge() + monkeypatch.setattr(viewer_module, "get_bridge", lambda: fake) + # `send_file` hands the work to `FileTransferSender`, which reads + # `get_bridge` out of `webrtc_files` -- patching it here alone left that + # path on the real bridge, so the chunks landed on a background event + # loop and the assertion read `sent[0]` before anything was in it. It + # passed wherever the loop won the race, which was every square until a + # loaded runner lost it. + from je_auto_control.utils.remote_desktop import webrtc_files + monkeypatch.setattr(webrtc_files, "get_bridge", lambda: fake) + return fake + + +@pytest.fixture +def viewer(): + """A viewer with an open control channel, as after `_attach_datachannel`.""" + instance = WebRTCDesktopViewer(token="secret") + channel = Channel() + instance._control_channel = channel + instance._wire_control_channel(channel) + channel.sent.clear() # drop the auth envelope the wiring may send + return instance + + +def _sent(viewer_instance): + return [json.loads(text) for text in viewer_instance._control_channel.sent] + + +def _deliver(viewer_instance, payload): + viewer_instance._control_channel.fire("message", json.dumps(payload)) + + +# --- outbound verbs ----------------------------------------------------------- + +def test_input_is_sent_as_a_payload_envelope(viewer): + viewer.send_input({"kind": "mouse", "x": 3}) + assert _sent(viewer) == [{"type": "input", + "payload": {"kind": "mouse", "x": 3}}] + + +def test_the_input_payload_is_copied_not_referenced(viewer): + payload = {"kind": "mouse"} + viewer.send_input(payload) + payload["kind"] = "mutated" + assert _sent(viewer)[0]["payload"] == {"kind": "mouse"} + + +@pytest.mark.parametrize("method,expected", [ + ("request_send_sas", "send_sas"), + ("request_inbox_listing", "list_inbox"), + ("request_renegotiation", "renegotiate_request"), +]) +def test_the_parameterless_verbs_each_send_their_envelope(viewer, method, + expected): + getattr(viewer, method)() + assert _sent(viewer) == [{"type": expected}] + + +def test_requesting_a_file_names_it(viewer): + viewer.request_inbox_file("report.txt") + assert _sent(viewer) == [{"type": "request_file", "name": "report.txt"}] + + +def test_deleting_a_file_names_it(viewer): + viewer.delete_inbox_file("report.txt") + assert _sent(viewer) == [{"type": "delete_inbox_file", + "name": "report.txt"}] + + +@pytest.mark.parametrize("method", ["request_inbox_file", "delete_inbox_file"]) +def test_an_empty_name_is_refused_before_it_reaches_the_wire(viewer, method): + with pytest.raises(ValueError, match="name required"): + getattr(viewer, method)("") + assert _sent(viewer) == [] + + +def test_sending_before_the_channel_opens_is_dropped_not_raised(bridge): + # The GUI can wire buttons up before the host's channel arrives; a click + # in that window is a no-op, not a traceback in a Qt slot. + WebRTCDesktopViewer(token="secret").send_input({"kind": "mouse"}) + assert bridge.deferred == [] + + +def test_a_channel_that_fails_mid_send_does_not_raise(viewer): + viewer._control_channel.send_error = OSError("channel closed") + viewer.send_input({"kind": "mouse"}) + + +def test_a_channel_closed_between_queue_and_send_is_a_no_op(viewer): + viewer._control_channel = None + viewer._safe_channel_send("{}") + + +def test_the_auth_envelope_omits_a_viewer_id_that_was_never_set(viewer): + viewer._send_auth() + assert _sent(viewer) == [{"type": "auth", "token": "secret"}] + + +# --- inbound envelopes: shape ------------------------------------------------- + +@pytest.mark.parametrize("message", [b"\x00", 42, None, ["auth_ok"]]) +def test_a_non_text_message_is_ignored(viewer, message): + viewer._control_channel.fire("message", message) + assert viewer.authenticated is False + + +def test_a_malformed_envelope_is_dropped(viewer): + viewer._control_channel.fire("message", "{not json") + assert viewer.authenticated is False + + +def test_a_json_scalar_is_not_an_envelope(viewer): + viewer._control_channel.fire("message", '"auth_ok"') + assert viewer.authenticated is False + + +def test_an_unknown_message_type_is_ignored(viewer): + _deliver(viewer, {"type": "shutdown"}) + assert viewer.authenticated is False + + +def test_closing_the_control_channel_ends_the_session(viewer): + viewer._authenticated = True + viewer._control_channel.fire("close") + assert viewer.authenticated is False + + +# --- authentication result ---------------------------------------------------- + +def test_auth_ok_makes_the_session_live(): + seen = [] + viewer = WebRTCDesktopViewer(token="secret", on_auth_result=seen.append) + viewer._control_channel = Channel() + viewer._wire_control_channel(viewer._control_channel) + _deliver(viewer, {"type": "auth_ok"}) + assert viewer.authenticated is True + assert seen == [True] + + +def test_auth_ok_carries_the_read_only_flag(viewer): + _deliver(viewer, {"type": "auth_ok", "read_only": True}) + assert viewer.read_only is True + + +def test_auth_ok_publishes_the_host_fingerprint(): + seen = [] + viewer = WebRTCDesktopViewer(token="secret", on_fingerprint=seen.append) + viewer._control_channel = Channel() + viewer._wire_control_channel(viewer._control_channel) + _deliver(viewer, {"type": "auth_ok", "fingerprint": "AB:CD:EF"}) + assert viewer.host_fingerprint == "AB:CD:EF" + assert seen == ["AB:CD:EF"] + + +def test_the_fingerprint_is_recorded_even_with_nobody_subscribed(viewer): + # `host_fingerprint` is read by the GUI on demand as well as pushed, so + # it has to be stored whether or not a callback was registered. + _deliver(viewer, {"type": "auth_ok", "fingerprint": "AB:CD:EF"}) + assert viewer.host_fingerprint == "AB:CD:EF" + + +@pytest.mark.parametrize("fingerprint", [None, "", 42]) +def test_an_absent_fingerprint_does_not_overwrite_the_one_on_screen( + viewer, fingerprint): + # It is the string the user compared out-of-band; a later envelope + # without one must leave it standing rather than blank the field. + viewer._host_fingerprint = "AB:CD:EF" + _deliver(viewer, {"type": "auth_ok", "fingerprint": fingerprint}) + assert viewer.host_fingerprint == "AB:CD:EF" + + +def test_a_raising_fingerprint_callback_still_authenticates(): + def _boom(_value): + raise RuntimeError("Qt widget already deleted") + + viewer = WebRTCDesktopViewer(token="secret", on_fingerprint=_boom) + viewer._control_channel = Channel() + viewer._wire_control_channel(viewer._control_channel) + _deliver(viewer, {"type": "auth_ok", "fingerprint": "AB"}) + assert viewer.authenticated is True + + +def test_auth_fail_leaves_the_session_closed(): + seen = [] + viewer = WebRTCDesktopViewer(token="wrong", on_auth_result=seen.append) + viewer._control_channel = Channel() + viewer._wire_control_channel(viewer._control_channel) + viewer._authenticated = True + _deliver(viewer, {"type": "auth_fail"}) + assert viewer.authenticated is False + assert seen == [False] + + +def test_an_auth_result_with_no_listener_is_survivable(viewer): + _deliver(viewer, {"type": "auth_ok"}) + assert viewer.authenticated is True + + +def test_a_raising_auth_callback_does_not_break_the_channel(): + def _boom(_ok): + raise RuntimeError("Qt widget already deleted") + + viewer = WebRTCDesktopViewer(token="secret", on_auth_result=_boom) + viewer._control_channel = Channel() + viewer._wire_control_channel(viewer._control_channel) + _deliver(viewer, {"type": "auth_ok"}) + assert viewer.authenticated is True + + +# --- permission updates ------------------------------------------------------- + +def test_the_legacy_read_only_envelope_is_still_understood(viewer): + _deliver(viewer, {"type": "read_only", "value": True}) + assert viewer.read_only is True + _deliver(viewer, {"type": "read_only", "value": False}) + assert viewer.read_only is False + + +def test_a_read_only_envelope_with_no_value_means_not_read_only(viewer): + viewer._read_only = True + _deliver(viewer, {"type": "read_only"}) + assert viewer.read_only is False + + +def test_the_permissions_envelope_mirrors_the_input_flag(viewer): + _deliver(viewer, {"type": "permissions", "value": {"allow_input": False}}) + assert viewer.read_only is True + _deliver(viewer, {"type": "permissions", "value": {"allow_input": True}}) + assert viewer.read_only is False + + +def test_a_permissions_envelope_without_the_input_flag_assumes_permitted( + viewer): + # The GUI greys out its own controls from this; defaulting to denied + # would lock the user out because a newer host sent a shorter dict. + viewer._read_only = True + _deliver(viewer, {"type": "permissions", "value": {"allow_files": True}}) + assert viewer.read_only is False + + +def test_a_permissions_envelope_that_is_not_a_dict_is_ignored(viewer): + viewer._read_only = True + _deliver(viewer, {"type": "permissions", "value": "read_only"}) + assert viewer.read_only is True + + +# --- the inbox listing -------------------------------------------------------- + +def test_an_inbox_listing_reaches_its_callback(viewer): + seen = [] + viewer.set_inbox_listing_callback(seen.append) + _deliver(viewer, {"type": "list_inbox_response", + "files": [{"name": "a.txt", "size": 1}]}) + assert seen == [[{"name": "a.txt", "size": 1}]] + + +def test_a_listing_with_no_files_key_becomes_an_empty_list(viewer): + seen = [] + viewer.set_inbox_listing_callback(seen.append) + _deliver(viewer, {"type": "list_inbox_response"}) + assert seen == [[]] + + +def test_a_listing_with_no_listener_is_dropped(viewer): + _deliver(viewer, {"type": "list_inbox_response", "files": []}) + + +def test_a_raising_listing_callback_is_contained(viewer): + def _boom(_files): + raise RuntimeError("Qt model already deleted") + + viewer.set_inbox_listing_callback(_boom) + _deliver(viewer, {"type": "list_inbox_response", "files": []}) + + +@pytest.mark.parametrize("msg_type", ["delete_inbox_response", + "request_file_response"]) +def test_an_inbox_operation_result_reaches_its_callback(viewer, msg_type): + seen = [] + viewer.set_inbox_op_result_callback( + lambda name, ok, error: seen.append((name, ok, error)), + ) + _deliver(viewer, {"type": msg_type, "name": "a.txt", "ok": False, + "error": "not found"}) + assert seen == [("a.txt", False, "not found")] + + +def test_an_operation_result_defaults_to_a_failure(viewer): + # A response missing its fields is not a success; the GUI would + # otherwise report a delete that never happened. + seen = [] + viewer.set_inbox_op_result_callback( + lambda name, ok, error: seen.append((name, ok, error)), + ) + _deliver(viewer, {"type": "delete_inbox_response"}) + assert seen == [("", False, None)] + + +def test_an_operation_result_with_no_listener_is_dropped(viewer): + _deliver(viewer, {"type": "delete_inbox_response", "name": "a.txt"}) + + +def test_a_raising_operation_callback_is_contained(viewer): + def _boom(_name, _ok, _error): + raise RuntimeError("Qt widget already deleted") + + viewer.set_inbox_op_result_callback(_boom) + _deliver(viewer, {"type": "delete_inbox_response", "name": "a.txt"}) + + +# --- host-initiated renegotiation --------------------------------------------- + +def test_a_renegotiate_offer_without_a_connection_is_ignored(viewer): + _deliver(viewer, {"type": "renegotiate_offer", "sdp": "v=0 offer"}) + assert viewer._background_tasks == set() + + +def test_a_renegotiate_offer_with_no_sdp_is_ignored(viewer): + viewer._pc = object() + _deliver(viewer, {"type": "renegotiate_offer", "sdp": None}) + assert viewer._background_tasks == set() + + +def test_a_renegotiate_offer_is_handled_on_the_event_loop(viewer, monkeypatch): + applied = [] + + class _Pc: + transceivers = [] + + async def setRemoteDescription(self, description): # noqa: N802 + applied.append(description.sdp) + + def getTransceivers(self): # noqa: N802 # reason: the aiortc name + return [] + + async def createAnswer(self): # noqa: N802 # reason: the aiortc name + return "answer" + + async def setLocalDescription(self, description): # noqa: N802 + pass + + @property + def localDescription(self): # noqa: N802 # reason: the aiortc name + return type("_Desc", (), {"sdp": "v=0 answer"})() + + async def _drive(): + viewer._pc = _Pc() + _deliver(viewer, {"type": "renegotiate_offer", "sdp": "v=0 offer"}) + assert len(viewer._background_tasks) == 1 + await asyncio.gather(*viewer._background_tasks) + + async def _noop(pc, timeout=None): + return None + + monkeypatch.setattr(viewer_module, "wait_for_ice_gathering", _noop) + asyncio.run(_drive()) + assert applied == ["v=0 offer"] + + +# --- microphone uplink -------------------------------------------------------- + +@pytest.fixture +def mic_sender(monkeypatch): + monkeypatch.setattr( + "je_auto_control.utils.remote_desktop.webrtc_mic.MicUplinkSender", + MicSender, + ) + + +def test_enabling_the_microphone_before_connecting_is_refused(viewer): + with pytest.raises(RuntimeError, match="mic channel not open"): + viewer.enable_mic_send() + + +def test_enabling_the_microphone_starts_one_sender(viewer, mic_sender): + channel = Channel("mic") + viewer._mic_channel = channel + viewer.enable_mic_send() + first = viewer._mic_sender + assert first.started + assert first.channel is channel + assert viewer.mic_active is True + viewer.enable_mic_send() + assert viewer._mic_sender is first + + +def test_disabling_the_microphone_stops_it(viewer, mic_sender): + viewer._mic_channel = Channel("mic") + viewer.enable_mic_send() + viewer.disable_mic_send() + assert viewer._mic_sender is None + assert viewer.mic_active is False + + +def test_disabling_a_microphone_that_was_never_enabled_is_a_no_op(viewer): + viewer.disable_mic_send() + assert viewer.mic_active is False + + +def test_a_microphone_that_fails_to_close_is_still_forgotten(viewer, + mic_sender): + viewer._mic_channel = Channel("mic") + viewer.enable_mic_send() + viewer._mic_sender.stop_error = OSError("stream already closed") + viewer.disable_mic_send() + assert viewer._mic_sender is None + + +def test_a_sender_that_stopped_on_its_own_is_not_active(viewer, mic_sender): + viewer._mic_channel = Channel("mic") + viewer.enable_mic_send() + viewer._mic_sender.running = False + assert viewer.mic_active is False + + +# --- files -------------------------------------------------------------------- + +def test_sending_a_file_before_the_channel_opens_is_refused(viewer): + with pytest.raises(RuntimeError, match="files channel not open"): + viewer.send_file("C:/report.txt") + + +def test_sending_a_file_streams_it_over_the_files_channel(viewer, tmp_path): + source = tmp_path / "notes.txt" + source.write_bytes(b"hello") + viewer._files_channel = Channel("files") + viewer.send_file(str(source), remote_name="renamed.txt") + envelope = json.loads(viewer._files_channel.sent[0]) + assert (envelope["type"], envelope["name"]) == ("file_begin", + "renamed.txt") + + +@pytest.fixture +def viewer_inbox(tmp_path, monkeypatch): + """Point the viewer's inbox at tmp_path. + + The viewer takes no `inbox_dir` -- unlike the host it always uses + `webrtc_files._DEFAULT_INBOX`, which is resolved at import time under + the real `~/.je_auto_control`. Redirecting HOME would be too late. + """ + from je_auto_control.utils.remote_desktop import webrtc_files + inbox = tmp_path / "inbox" + monkeypatch.setattr(webrtc_files, "_DEFAULT_INBOX", inbox) + return inbox + + +def test_a_file_pushed_by_the_host_lands_and_is_announced(viewer, + viewer_inbox): + seen = [] + viewer.set_file_received_callback(seen.append) + channel = Channel("files") + viewer._wire_files_channel(channel) + inbox = viewer_inbox + channel.fire("message", json.dumps({"type": "file_begin", + "name": "pushed.txt", "size": 4})) + channel.fire("message", b"data") + channel.fire("message", json.dumps({"type": "file_end"})) + assert (inbox / "pushed.txt").read_bytes() == b"data" + assert seen and seen[0].name == "pushed.txt" + + +def test_a_file_arriving_with_no_listener_is_still_written(viewer, + viewer_inbox): + channel = Channel("files") + viewer._wire_files_channel(channel) + channel.fire("message", json.dumps({"type": "file_begin", + "name": "pushed.txt", "size": 1})) + channel.fire("message", b"x") + channel.fire("message", json.dumps({"type": "file_end"})) + assert (viewer._files_receiver._inbox / "pushed.txt").exists() + + +def test_a_raising_file_callback_does_not_lose_the_file(viewer): + def _boom(_path): + raise RuntimeError("Qt widget already deleted") + + viewer.set_file_received_callback(_boom) + viewer._on_viewer_file_done("C:/inbox/pushed.txt") + + +def test_the_files_receiver_is_built_once_and_reused(viewer, viewer_inbox): + viewer._wire_files_channel(Channel("files")) + first = viewer._files_receiver + viewer._wire_files_channel(Channel("files")) + assert viewer._files_receiver is first diff --git a/test/unit_test/headless/test_webrtc_viewer_media.py b/test/unit_test/headless/test_webrtc_viewer_media.py new file mode 100644 index 00000000..1fcbfebe --- /dev/null +++ b/test/unit_test/headless/test_webrtc_viewer_media.py @@ -0,0 +1,385 @@ +"""Attaching the viewer's own screen and microphone to the host's slots. + +Split out of `test_webrtc_viewer_session.py`, which covers the offer/answer +exchange this rides on. Everything here turns on one upstream fact and one +design decision: + +* **Slots are identified by m-line order, not by direction.** aiortc gives + every answerer transceiver the default `recvonly` direction regardless of + what the offer asked for, so the viewer cannot filter by direction and + counts instead: the *second* video transceiver is the host's recvonly + slot, because the first is the host's own outbound screen. Off by one + there and the viewer replaces the picture it is here to watch. +* **OFF is in-place; ON renegotiates.** Turning a stream off is + `replaceTrack(None)` plus a stop, keeping the SDP direction so the slot + survives -- the host simply sees its consume task end. Turning one on + always needs a fresh `track` event at the host, so it costs a + renegotiation round trip. Getting that backwards costs one on every mute. + +The doubles come from `headless._webrtc_doubles`; what is not faked is the +slot arithmetic, which is the thing under test. +""" +from __future__ import annotations + +import asyncio + +import pytest + +from headless._webrtc_doubles import ( + Bridge, Channel, FakePeerConnection, Track, Transceiver, + noop_ice_gathering, +) +from je_auto_control.utils.remote_desktop import webrtc_viewer as viewer_module +from je_auto_control.utils.remote_desktop.webrtc_transport import WebRTCConfig +from je_auto_control.utils.remote_desktop.webrtc_viewer import ( + WebRTCDesktopViewer, +) + + +@pytest.fixture(autouse=True) +def fake_peer_connection(monkeypatch): + FakePeerConnection.instances = [] + monkeypatch.setattr(viewer_module, "RTCPeerConnection", FakePeerConnection) + monkeypatch.setattr(viewer_module, "wait_for_ice_gathering", + noop_ice_gathering) + yield + FakePeerConnection.instances = [] + + +@pytest.fixture(autouse=True) +def bridge(monkeypatch): + fake = Bridge() + monkeypatch.setattr(viewer_module, "get_bridge", lambda: fake) + return fake + + +@pytest.fixture +def screen_track(monkeypatch): + """Replace the capture the viewer attaches when it shares its screen.""" + made = [] + + def _factory(**kwargs): + track = Track("video") + track.kwargs = kwargs + made.append(track) + return track + + monkeypatch.setattr( + "je_auto_control.utils.remote_desktop.webrtc_transport" + ".ScreenVideoTrack", _factory, + ) + return made + + +@pytest.fixture +def opus_track(monkeypatch): + made = [] + + def _factory(): + track = Track("audio") + made.append(track) + return track + + monkeypatch.setattr( + "je_auto_control.utils.remote_desktop.webrtc_audio.OpusMicAudioTrack", + _factory, + ) + return made + + +def _viewer(**kwargs) -> WebRTCDesktopViewer: + kwargs.setdefault("token", "secret") + return WebRTCDesktopViewer(**kwargs) + + +def _connected(**kwargs): + """A viewer with a peer connection already in place.""" + viewer = _viewer(**kwargs) + viewer._pc = FakePeerConnection() + return viewer, viewer._pc + + +# --- attaching the viewer's own media ----------------------------------------- + +def _video_slots(pc, count=2): + pc.transceivers = [Transceiver("video") for _ in range(count)] + return pc.transceivers + + +def test_sharing_a_screen_takes_the_second_video_slot(screen_track): + # The first video m-line is the host's outbound screen; taking it would + # replace the picture the viewer is here to watch. + viewer = _viewer(config=WebRTCConfig(share_my_screen=True)) + viewer.process_offer("v=0 host-offer") + pc = FakePeerConnection.instances[0] + assert pc.transceivers == [] # no slots offered + viewer._pc = pc + first, second = _video_slots(pc) + viewer._attach_viewer_screen_track() + assert second.sender.track is screen_track[0] + assert second.direction == "sendonly" + assert first.sender.track is None + + +def test_sharing_a_screen_uses_the_viewers_own_capture_settings(screen_track): + config = WebRTCConfig(share_my_screen=True, monitor_index=2, fps=12, + region=(1, 2, 3, 4), show_cursor=False) + viewer, pc = _connected(config=config) + _video_slots(pc) + viewer._attach_viewer_screen_track() + assert screen_track[0].kwargs == { + "monitor_index": 2, "fps": 12, "region": (1, 2, 3, 4), + "show_cursor": False, + } + + +def test_sharing_a_screen_the_host_never_offered_is_a_warning_not_a_crash( + screen_track): + # `accept_viewer_video=False` on the host means there is no second slot; + # the viewer must keep watching rather than raise out of the answer. + viewer, pc = _connected(config=WebRTCConfig(share_my_screen=True)) + _video_slots(pc, count=1) + viewer._attach_viewer_screen_track() + assert viewer._viewer_screen_track is None + assert screen_track == [] + + +def test_sharing_a_microphone_takes_the_only_audio_slot(opus_track): + viewer, pc = _connected(config=WebRTCConfig(share_my_audio_opus=True)) + slot = Transceiver("audio") + pc.transceivers = [Transceiver("video"), slot] + viewer._attach_opus_audio_track() + assert slot.sender.track is opus_track[0] + assert slot.direction == "sendonly" + + +def test_sharing_a_microphone_the_host_never_offered_is_survivable(opus_track): + viewer, pc = _connected(config=WebRTCConfig(share_my_audio_opus=True)) + pc.transceivers = [Transceiver("video")] + viewer._attach_opus_audio_track() + assert viewer._opus_audio_track is None + + +def test_a_viewer_with_no_microphone_still_answers(monkeypatch): + def _no_device(): + raise OSError("no input device") + + monkeypatch.setattr( + "je_auto_control.utils.remote_desktop.webrtc_audio.OpusMicAudioTrack", + _no_device, + ) + viewer, pc = _connected(config=WebRTCConfig(share_my_audio_opus=True)) + pc.transceivers = [Transceiver("audio")] + viewer._attach_opus_audio_track() + assert viewer._opus_audio_track is None + + +def test_the_answer_attaches_both_streams_the_config_asks_for(screen_track, + opus_track): + config = WebRTCConfig(share_my_screen=True, share_my_audio_opus=True) + viewer = _viewer(config=config) + + class _PcWithSlots(FakePeerConnection): + def __init__(self, configuration=None) -> None: + super().__init__(configuration) + self.transceivers = [Transceiver("video"), Transceiver("video"), + Transceiver("audio")] + + viewer_module.RTCPeerConnection = _PcWithSlots + try: + viewer.process_offer("v=0 host-offer") + finally: + viewer_module.RTCPeerConnection = FakePeerConnection + assert viewer._viewer_screen_track is screen_track[0] + assert viewer._opus_audio_track is opus_track[0] + + +# --- live toggles ------------------------------------------------------------- + +def test_turning_screen_share_on_asks_the_host_to_renegotiate(): + viewer, _ = _connected() + channel = Channel() + viewer._control_channel = channel + viewer.toggle_share_screen(True) + assert viewer._config.share_my_screen is True + assert '"renegotiate_request"' in channel.sent[0] + + +def test_turning_screen_share_on_twice_costs_one_renegotiation(): + viewer, _ = _connected() + channel = Channel() + viewer._control_channel = channel + viewer._viewer_screen_track = Track("video") + viewer.toggle_share_screen(True) + assert channel.sent == [], "already sharing" + + +def test_turning_screen_share_off_detaches_in_place(): + # OFF costs nothing: the SDP direction stays, so the slot survives and + # the host simply sees its consume task end. + viewer, pc = _connected() + channel = Channel() + viewer._control_channel = channel + track = Track("video") + viewer._viewer_screen_track = track + pc.transceivers = [Transceiver("video", track=track)] + viewer.toggle_share_screen(False) + assert track.stopped + assert pc.transceivers[0].sender.replaced == [None] + assert viewer._viewer_screen_track is None + assert channel.sent == [], "no renegotiation on the way down" + + +def test_turning_the_microphone_off_detaches_in_place(): + viewer, pc = _connected() + track = Track("audio") + viewer._opus_audio_track = track + pc.transceivers = [Transceiver("audio", track=track)] + viewer.toggle_opus_mic(False) + assert track.stopped + assert viewer._opus_audio_track is None + + +def test_turning_the_microphone_on_asks_the_host_to_renegotiate(): + viewer, _ = _connected() + channel = Channel() + viewer._control_channel = channel + viewer.toggle_opus_mic(True) + assert viewer._config.share_my_audio_opus is True + assert '"renegotiate_request"' in channel.sent[0] + + +def test_turning_the_microphone_on_twice_costs_one_renegotiation(): + viewer, _ = _connected() + channel = Channel() + viewer._control_channel = channel + viewer._opus_audio_track = Track("audio") + viewer.toggle_opus_mic(True) + assert channel.sent == [] + + +def test_detaching_skips_transceivers_that_hold_a_different_track(): + # Two video slots and only one of them is ours; the other is the host's + # screen, and detaching it would blank the window. + viewer, pc = _connected() + ours = Track("video") + theirs = Transceiver("video", track=Track("video")) + mine = Transceiver("video", track=ours) + pc.transceivers = [theirs, mine] + viewer._viewer_screen_track = ours + viewer.toggle_share_screen(False) + assert theirs.sender.replaced == [] + assert mine.sender.replaced == [None] + + +def test_detaching_ignores_transceivers_of_the_other_kind(): + viewer, pc = _connected() + track = Track("audio") + audio_slot = Transceiver("audio", track=track) + pc.transceivers = [Transceiver("video", track=track), audio_slot] + viewer._opus_audio_track = track + viewer.toggle_opus_mic(False) + assert audio_slot.sender.replaced == [None] + + +def test_a_track_the_connection_no_longer_carries_is_still_stopped(): + # Renegotiation can replace the transceiver set underneath us; the + # capture thread has to be stopped even when its slot has gone. + viewer, pc = _connected() + ours = Track("video") + pc.transceivers = [Transceiver("video", track=Track("video"))] + viewer._viewer_screen_track = ours + viewer.toggle_share_screen(False) + assert ours.stopped + assert viewer._viewer_screen_track is None + + +def test_detaching_a_track_that_was_never_attached_is_a_no_op(): + viewer, pc = _connected() + pc.transceivers = [Transceiver("video")] + viewer.toggle_share_screen(False) + assert pc.transceivers[0].sender.replaced == [] + + +def test_detaching_without_a_peer_connection_is_a_no_op(): + viewer = _viewer() + viewer._viewer_screen_track = Track("video") + viewer.toggle_share_screen(False) + assert viewer._viewer_screen_track is not None, "nothing to detach from" + + +def test_a_sender_that_rejects_the_detach_still_stops_the_capture(): + # A PeerConnection torn down underneath us fails `replaceTrack`; the + # capture thread must still be stopped or it runs until the process ends. + viewer, pc = _connected() + track = Track("video") + slot = Transceiver("video", track=track) + slot.sender.replace_error = RuntimeError("connection closed") + pc.transceivers = [slot] + viewer._viewer_screen_track = track + viewer.toggle_share_screen(False) + assert track.stopped + assert viewer._viewer_screen_track is None + + +def test_a_capture_that_fails_to_stop_is_still_forgotten(): + viewer, pc = _connected() + + class _StubbornTrack(Track): + def stop(self): + raise OSError("device gone") + + track = _StubbornTrack("video") + pc.transceivers = [Transceiver("video", track=track)] + viewer._viewer_screen_track = track + viewer.toggle_share_screen(False) + assert viewer._viewer_screen_track is None + + +# --- host-initiated renegotiation --------------------------------------------- + +def test_a_renegotiation_offer_is_answered_over_the_control_channel(): + viewer, pc = _connected() + channel = Channel() + viewer._control_channel = channel + asyncio.run(viewer._async_handle_renegotiate("v=0 new-offer")) + assert pc.remote_descriptions[0].type == "offer" + assert '"renegotiate_answer"' in channel.sent[0] + assert "v=0 local-sdp" in channel.sent[0] + + +def test_a_renegotiation_attaches_the_media_the_toggles_asked_for( + screen_track, opus_track): + config = WebRTCConfig(share_my_screen=True, share_my_audio_opus=True) + viewer, pc = _connected(config=config) + viewer._control_channel = Channel() + pc.transceivers = [Transceiver("video"), Transceiver("video"), + Transceiver("audio")] + asyncio.run(viewer._async_handle_renegotiate("v=0 new-offer")) + assert viewer._viewer_screen_track is screen_track[0] + assert viewer._opus_audio_track is opus_track[0] + + +def test_a_renegotiation_does_not_re_attach_media_already_sending( + screen_track): + viewer, pc = _connected(config=WebRTCConfig(share_my_screen=True)) + viewer._control_channel = Channel() + existing = Track("video") + viewer._viewer_screen_track = existing + pc.transceivers = [Transceiver("video"), Transceiver("video")] + asyncio.run(viewer._async_handle_renegotiate("v=0 new-offer")) + assert viewer._viewer_screen_track is existing + assert screen_track == [] + + +def test_a_renegotiation_aiortc_rejects_sends_no_answer(): + viewer, pc = _connected() + channel = Channel() + viewer._control_channel = channel + pc.answer_error = RuntimeError("invalid SDP") + asyncio.run(viewer._async_handle_renegotiate("v=0 nonsense")) + assert channel.sent == [], "a half-built answer is worse than none" + + +def test_a_renegotiation_without_a_peer_connection_is_a_no_op(): + asyncio.run(_viewer()._async_handle_renegotiate("v=0 new-offer")) diff --git a/test/unit_test/headless/test_webrtc_viewer_session.py b/test/unit_test/headless/test_webrtc_viewer_session.py new file mode 100644 index 00000000..4e8cad5c --- /dev/null +++ b/test/unit_test/headless/test_webrtc_viewer_session.py @@ -0,0 +1,465 @@ +"""Answering a host's offer, watching what arrives, and letting go. + +`WebRTCDesktopViewer` is the offer-consumer half of the pair. Like the host +it needs aiortc to import at all, so nothing here ran on a CI square before +the `[webrtc]` extra joined the measured install. It is split three ways: +the media it attaches to the host's slots is in +`test_webrtc_viewer_media.py`, the control channel in +`test_webrtc_viewer_control.py`, and this file has the session around them. + +Three things here are load-bearing well beyond their size: + +* **The fingerprint is checked before any DTLS handshake.** That check is + the whole value of the pinning feature -- a signaling slot that has been + hijacked has to be caught while the offer is still text, not after + encrypted bytes are flowing, so a mismatch must not even build a + PeerConnection. +* **Inbound channels are routed by label**, and the control channel may + arrive already open. aiortc fires "datachannel" whenever it likes; if the + channel is open by then, "open" never fires again and an authentication + handshake that waited for it would wait forever. +* **The frame pump has to end quietly.** Nobody awaits it, so every way a + track can stop -- the host ending its share, the transport failing, the + viewer closing -- is a path that must not leave an exception behind. + +The doubles come from `headless._webrtc_doubles`. +""" +from __future__ import annotations + +import asyncio + +import pytest + +from headless._webrtc_doubles import ( + Bridge, Channel, FakePeerConnection, FrameTrack, HangingBridge, + Stoppable, Track, noop_ice_gathering, +) +from je_auto_control.utils.remote_desktop import webrtc_viewer as viewer_module +from je_auto_control.utils.remote_desktop.fingerprint import ( + FingerprintMismatchError, +) +from je_auto_control.utils.remote_desktop.webrtc_transport import WebRTCConfig +from je_auto_control.utils.remote_desktop.webrtc_viewer import ( + WebRTCDesktopViewer, +) + + +@pytest.fixture(autouse=True) +def fake_peer_connection(monkeypatch): + FakePeerConnection.instances = [] + monkeypatch.setattr(viewer_module, "RTCPeerConnection", FakePeerConnection) + monkeypatch.setattr(viewer_module, "wait_for_ice_gathering", + noop_ice_gathering) + yield + FakePeerConnection.instances = [] + + +@pytest.fixture(autouse=True) +def bridge(monkeypatch): + fake = Bridge() + monkeypatch.setattr(viewer_module, "get_bridge", lambda: fake) + return fake + + +@pytest.fixture +def screen_track(monkeypatch): + """Replace the capture the viewer attaches when it shares its screen.""" + made = [] + + def _factory(**kwargs): + track = Track("video") + track.kwargs = kwargs + made.append(track) + return track + + monkeypatch.setattr( + "je_auto_control.utils.remote_desktop.webrtc_transport" + ".ScreenVideoTrack", _factory, + ) + return made + + +@pytest.fixture +def opus_track(monkeypatch): + made = [] + + def _factory(): + track = Track("audio") + made.append(track) + return track + + monkeypatch.setattr( + "je_auto_control.utils.remote_desktop.webrtc_audio.OpusMicAudioTrack", + _factory, + ) + return made + + +def _viewer(**kwargs) -> WebRTCDesktopViewer: + kwargs.setdefault("token", "secret") + return WebRTCDesktopViewer(**kwargs) + + +def _connected(**kwargs): + """A viewer with a peer connection already in place.""" + viewer = _viewer(**kwargs) + viewer._pc = FakePeerConnection() + return viewer, viewer._pc + + +# --- construction ------------------------------------------------------------- + +def test_a_viewer_without_a_token_is_refused(): + with pytest.raises(ValueError, match="non-empty token"): + WebRTCDesktopViewer(token="") + + +def test_a_fresh_viewer_reports_no_session(): + viewer = _viewer() + assert viewer.authenticated is False + assert viewer.read_only is False + assert viewer.host_fingerprint is None + assert viewer.connection_state == "closed" + + +def test_asking_for_the_peer_connection_before_connecting_is_an_error(): + with pytest.raises(RuntimeError, match="connect first"): + _viewer()._require_pc() + + +# --- processing the offer ----------------------------------------------------- + +@pytest.mark.parametrize("offer", ["", " "]) +def test_an_empty_offer_is_refused_before_it_reaches_aiortc(offer): + with pytest.raises(ValueError, match="offer_sdp is empty"): + _viewer().process_offer(offer) + + +def test_processing_an_offer_returns_the_answer_sdp(): + assert _viewer().process_offer("v=0 host-offer") == "v=0 local-sdp" + + +def test_the_offer_is_applied_as_the_remote_description(): + viewer = _viewer() + viewer.process_offer("v=0 host-offer") + [pc] = FakePeerConnection.instances + [description] = pc.remote_descriptions + assert (description.sdp, description.type) == ("v=0 host-offer", "offer") + + +def test_the_answer_uses_the_viewers_own_ice_servers(): + config = WebRTCConfig(ice_servers=["stun:example:3478"]) + _viewer(config=config).process_offer("v=0 host-offer") + [pc] = FakePeerConnection.instances + assert [s.urls for s in pc.configuration.iceServers] == [ + "stun:example:3478", + ] + + +def test_a_second_offer_closes_the_first_peer_connection(): + viewer = _viewer() + viewer.process_offer("v=0 host-offer") + viewer.process_offer("v=0 host-offer-2") + first, second = FakePeerConnection.instances + assert first.closed + assert not second.closed + + +def test_a_pinned_fingerprint_that_matches_lets_the_offer_through(): + sdp = "v=0\r\na=fingerprint:sha-256 AB:CD\r\n" + assert _viewer().process_offer(sdp, "abcd") == "v=0 local-sdp" + + +def test_a_pinned_fingerprint_that_does_not_match_stops_the_handshake(): + # The point of pinning is catching a hijacked signaling slot while the + # offer is still text -- so nothing may be built before the check. + sdp = "v=0\r\na=fingerprint:sha-256 AB:CD\r\n" + with pytest.raises(FingerprintMismatchError): + _viewer().process_offer(sdp, "ffff") + assert not FakePeerConnection.instances + + +def test_an_offer_with_no_fingerprint_at_all_is_refused_when_pinned(): + with pytest.raises(FingerprintMismatchError): + _viewer().process_offer("v=0\r\nm=video 9 UDP/TLS/RTP/SAVPF\r\n", "ab") + + +# --- peer connection events --------------------------------------------------- + +def test_the_state_callback_sees_every_transition(): + seen = [] + viewer = _viewer(on_state_change=seen.append) + viewer.process_offer("v=0 host-offer") + pc = FakePeerConnection.instances[0] + for state in ("connecting", "connected", "closed"): + pc.connectionState = state + asyncio.run(pc.fire("connectionstatechange")) + assert seen == ["connecting", "connected", "closed"] + assert viewer.connection_state == "closed" + + +@pytest.mark.parametrize("state", ["failed", "closed", "disconnected"]) +def test_a_lost_connection_drops_the_authenticated_flag(state): + viewer = _viewer() + viewer.process_offer("v=0 host-offer") + viewer._authenticated = True + pc = FakePeerConnection.instances[0] + pc.connectionState = state + asyncio.run(pc.fire("connectionstatechange")) + assert viewer.authenticated is False + + +def test_a_raising_state_callback_does_not_break_the_handler(): + def _boom(_state): + raise RuntimeError("Qt widget already deleted") + + viewer = _viewer(on_state_change=_boom) + viewer.process_offer("v=0 host-offer") + pc = FakePeerConnection.instances[0] + pc.connectionState = "connected" + asyncio.run(pc.fire("connectionstatechange")) + + +def test_an_inbound_video_track_starts_the_frame_pump(): + viewer = _viewer() + viewer.process_offer("v=0 host-offer") + + async def _drive(): + FakePeerConnection.instances[0].fire("track", Track("video")) + assert viewer._receive_task is not None + viewer._receive_task.cancel() + + asyncio.run(_drive()) + + +def test_an_inbound_audio_track_starts_host_voice_playback(monkeypatch): + receivers = [] + + class _Receiver: + def __init__(self) -> None: + self.consumed = [] + receivers.append(self) + + def consume(self, track): + self.consumed.append(track) + + monkeypatch.setattr( + "je_auto_control.utils.remote_desktop.webrtc_audio.OpusMicReceiver", + _Receiver, + ) + viewer = _viewer() + viewer.process_offer("v=0 host-offer") + track = Track("audio") + FakePeerConnection.instances[0].fire("track", track) + assert receivers[0].consumed == [track] + + +def test_a_second_audio_track_does_not_open_a_second_player(monkeypatch): + class _Receiver: + def consume(self, track): + pass + + monkeypatch.setattr( + "je_auto_control.utils.remote_desktop.webrtc_audio.OpusMicReceiver", + _Receiver, + ) + viewer = _viewer() + viewer.process_offer("v=0 host-offer") + pc = FakePeerConnection.instances[0] + pc.fire("track", Track("audio")) + first = viewer._host_voice_receiver + pc.fire("track", Track("audio")) + assert viewer._host_voice_receiver is first + + +def test_a_viewer_with_no_speaker_keeps_the_video(monkeypatch): + def _no_device(): + raise OSError("no output device") + + monkeypatch.setattr( + "je_auto_control.utils.remote_desktop.webrtc_audio.OpusMicReceiver", + _no_device, + ) + viewer = _viewer() + viewer.process_offer("v=0 host-offer") + FakePeerConnection.instances[0].fire("track", Track("audio")) + assert viewer._host_voice_receiver is None + + +def test_a_track_that_is_neither_audio_nor_video_is_ignored(): + viewer = _viewer() + viewer.process_offer("v=0 host-offer") + FakePeerConnection.instances[0].fire("track", Track("application")) + assert viewer._receive_task is None + assert viewer._host_voice_receiver is None + + +def test_frames_reach_the_paint_callback(): + seen = [] + viewer = _viewer(on_frame=seen.append) + asyncio.run(viewer._consume_video(FrameTrack("f1", "f2"))) + assert seen == ["f1", "f2"] + + +def test_the_end_of_the_stream_is_an_ending_not_an_escaping_exception(): + # `MediaStreamError` is how aiortc says "the host stopped sharing" -- the + # ordinary way a session ends. It derives straight from Exception, so it + # is not covered by the OSError / RuntimeError arm, and nobody awaits + # this task: letting it out turns every clean disconnect into an + # un-retrieved task exception. The host's drain loop always caught it; + # this one did not until 2026-08-24. + from aiortc.mediastreams import MediaStreamError + viewer = _viewer(on_frame=lambda frame: None) + asyncio.run(viewer._consume_video( + FrameTrack("f1", ending=MediaStreamError()), + )) + + +def test_a_transport_failure_also_ends_the_stream_quietly(): + viewer = _viewer(on_frame=lambda frame: None) + asyncio.run(viewer._consume_video(FrameTrack(ending=OSError("reset")))) + + +def test_a_raising_paint_callback_does_not_end_the_stream(): + seen = [] + + def _cb(frame): + seen.append(frame) + raise RuntimeError("QImage conversion failed") + + viewer = _viewer(on_frame=_cb) + asyncio.run(viewer._consume_video(FrameTrack("f1", "f2"))) + assert seen == ["f1", "f2"] + + +def test_frames_with_no_listener_are_still_drained(): + track = FrameTrack("f1", "f2") + asyncio.run(_viewer()._consume_video(track)) + assert track.frames == [] + + +def test_the_frame_pump_stops_when_the_viewer_is_closing(): + viewer = _viewer(on_frame=lambda frame: None) + viewer._closed.set() + track = FrameTrack("f1") + asyncio.run(viewer._consume_video(track)) + assert track.frames == ["f1"], "it never pulled a frame" + + +# --- data channel routing ----------------------------------------------------- + +@pytest.mark.parametrize("label,attribute", [ + ("mic", "_mic_channel"), + ("files", "_files_channel"), + ("usb", "_usb_channel"), + ("ctrl", "_control_channel"), +]) +def test_each_inbound_channel_is_routed_by_its_label(label, attribute): + viewer = _viewer() + viewer.process_offer("v=0 host-offer") + channel = Channel(label) + FakePeerConnection.instances[0].fire("datachannel", channel) + assert getattr(viewer, attribute) is channel + + +def test_an_unknown_label_is_treated_as_the_control_channel(): + # The host names it "ctrl"; anything else arriving is still the channel + # the auth handshake has to go out on, so it must not be dropped. + viewer = _viewer() + channel = Channel("control") + viewer._attach_datachannel(channel) + assert viewer._control_channel is channel + + +def test_the_control_channel_sends_auth_as_soon_as_it_opens(): + viewer = _viewer(viewer_id="viewer-3") + channel = Channel("ctrl") + viewer._attach_datachannel(channel) + assert channel.sent == [], "not open yet" + channel.fire("open") + assert '"auth"' in channel.sent[0] + assert '"viewer-3"' in channel.sent[0] + + +def test_a_channel_already_open_on_arrival_still_authenticates(): + # aiortc may fire "datachannel" after the channel is open, in which case + # the "open" event never fires again and the handshake would never start. + viewer = _viewer() + viewer._attach_datachannel(Channel("ctrl", ready_state="open")) + assert '"auth"' in viewer._control_channel.sent[0] + + +def test_the_usb_channel_gets_a_passthrough_client(): + viewer = _viewer() + viewer._attach_datachannel(Channel("usb")) + assert viewer.usb_client() is not None + + +def test_a_viewer_with_no_usb_channel_has_no_client(): + assert _viewer().usb_client() is None + + +# --- teardown ----------------------------------------------------------------- + +def test_stopping_a_viewer_that_never_connected_is_a_no_op(bridge): + _viewer().stop() + assert bridge.deferred == [] + + +def test_stop_closes_the_connection_and_forgets_the_channels(): + viewer = _viewer() + viewer.process_offer("v=0 host-offer") + viewer._control_channel = Channel() + viewer._files_channel = Channel("files") + viewer._authenticated = True + viewer.stop() + assert FakePeerConnection.instances[0].closed + assert viewer._pc is None + assert viewer._control_channel is None + assert viewer._files_channel is None + assert viewer.authenticated is False + + +def test_stop_releases_every_stream_the_viewer_owns(): + viewer = _viewer() + voice, opus, screen, mic = (Stoppable() for _ in range(4)) + viewer._host_voice_receiver = voice + viewer._opus_audio_track = opus + viewer._viewer_screen_track = screen + viewer._mic_sender = mic + asyncio.run(viewer._async_stop()) + assert all(s.stopped for s in (voice, opus, screen, mic)) + assert viewer._viewer_screen_track is None + + +def test_stop_cancels_the_frame_pump(): + viewer = _viewer() + + async def _drive(): + task = asyncio.ensure_future(asyncio.sleep(10)) + viewer._receive_task = task + await viewer._async_stop() + return task + + task = asyncio.run(_drive()) + assert task.cancelled() + assert viewer._receive_task is None + + +def test_a_teardown_failure_does_not_abort_the_rest_of_the_teardown(): + viewer = _viewer() + failing = Stoppable(error=OSError("device gone")) + rest = Stoppable() + viewer._host_voice_receiver = failing + viewer._opus_audio_track = rest + asyncio.run(viewer._async_stop()) + assert rest.stopped + + +def test_a_stop_that_times_out_is_reported_rather_than_raised(monkeypatch): + monkeypatch.setattr(viewer_module, "get_bridge", HangingBridge) + viewer = _viewer() + viewer._pc = object() + viewer.stop() + + diff --git a/test/unit_test/headless/test_window_backend_macos.py b/test/unit_test/headless/test_window_backend_macos.py new file mode 100644 index 00000000..f5c7e621 --- /dev/null +++ b/test/unit_test/headless/test_window_backend_macos.py @@ -0,0 +1,492 @@ +"""macOS window management, where reading and acting are two APIs. + +`macos_backend.py` was at 0% on every square. Like the X11 one it imports its +platform libraries inside its methods, so stubs in `sys.modules` +(`_pyobjc_stub.py`) exercise it from all nine rather than the two Darwin +squares -- and the coverage floor is the lowest square, so that difference is +the whole point. + +The split down the middle of this backend is what the tests are about: + +* **Quartz reads, the accessibility API acts**, and they have different + permission stories. Listing, rectangles and ownership need no grant; + moving, closing, minimising and raising are gated behind TCC, and until the + user grants Accessibility every one of them silently does nothing. So each + action reports refusal rather than a false success. +* **The two APIs do not share a handle.** A `CGWindowID` has no public bridge + to an `AXUIElement`, so the window is found again inside its owning + application by origin and then by title. Origin is the stronger signal -- + two windows cannot share one at the same moment, while titles are empty or + duplicated all the time -- and getting that preference backwards moves the + wrong window of an application that has several. +* **An AX call returns an error code, and zero is success.** `close()` and + `minimize()` return `not error`. Reading that backwards is a "yes" for + every failed action, which is exactly the answer a caller cannot recover + from. +* **Layer 0 is the application layer.** Menu bars, the Dock and status items + live above it and are not what anyone means by "the Safari window". + +`test_pyobjc_stub_names.py` holds the stub's surface to the real frameworks +wherever they are installed, which on CI is the macOS squares. +""" +from __future__ import annotations + +import pytest + +from headless import _pyobjc_stub as objc_stub +from headless._pyobjc_stub import AX_FAILURE, AXElement, World, window_info +from je_auto_control.utils.exception.exceptions import ( + AutoControlUnsupportedOperationException, +) +from je_auto_control.wrapper.window_backends import macos_backend as macos +from je_auto_control.wrapper.window_backends.macos_backend import ( + MacOSWindowBackend, _point, +) + + +@pytest.fixture +def on_darwin(monkeypatch): + monkeypatch.setattr(macos.sys, "platform", "darwin") + + +def _build(monkeypatch, world: World) -> MacOSWindowBackend: + objc_stub.install(monkeypatch, world) + backend = MacOSWindowBackend() + assert backend.available, "the stubbed frameworks import" + return backend + + +@pytest.fixture +def world(): + return World() + + +@pytest.fixture +def backend(monkeypatch, on_darwin, world): + return _build(monkeypatch, world) + + +# --- availability ------------------------------------------------------------- + +def test_the_backend_is_unavailable_off_macos(monkeypatch, world): + monkeypatch.setattr(macos.sys, "platform", "linux") + objc_stub.install(monkeypatch, world) + assert MacOSWindowBackend().available is False + + +def test_a_mac_without_pyobjc_is_unavailable_rather_than_fatal(monkeypatch, + on_darwin): + # pyobjc is a hard dependency on Darwin, but a broken or partial install + # must degrade to the null backend, not to an ImportError at start-up. + objc_stub.install_missing(monkeypatch, "Quartz") + assert MacOSWindowBackend().available is False + + +def test_the_backend_names_both_apis_it_uses(backend): + assert backend.name == "macos-quartz-ax" + + +# --- listing ------------------------------------------------------------------ + +def test_listing_asks_quartz_for_on_screen_windows_only(backend, world): + backend.list_windows() + [(options, relative_to)] = world.list_options + assert options == 1 | 16, "on-screen only, excluding desktop elements" + assert relative_to == 0, "kCGNullWindowID" + + +def test_listing_keeps_the_order_quartz_gives(backend, world): + # Quartz returns front-to-back, which is already what a caller means by + # "the first window that matches". + world.windows = [window_info(3, name="front"), window_info(1, name="back")] + assert backend.list_windows() == [(3, "front"), (1, "back")] + + +def test_listing_skips_everything_above_the_application_layer(backend, world): + world.windows = [window_info(1, name="Menubar", layer=25), + window_info(2, name="Editor", layer=0)] + assert backend.list_windows() == [(2, "Editor")] + + +def test_listing_skips_a_window_with_no_id(backend, world): + # Quartz omits the number for windows a caller has no business addressing. + world.windows = [window_info(0, name="anonymous")] + assert backend.list_windows() == [] + + +def test_a_window_with_no_title_lists_with_an_empty_one(backend, world): + # Screen-recording permission is what puts titles in this list; without + # it Quartz still reports the windows, with the names left out. + world.windows = [{"kCGWindowNumber": 7, "kCGWindowLayer": 0}] + assert backend.list_windows() == [(7, "")] + + +def test_a_desktop_quartz_answers_nothing_for_lists_nothing(backend, world): + world.windows = [] + assert backend.list_windows() == [] + + +# --- the focused window ------------------------------------------------------- + +def test_the_foreground_window_is_the_front_one_of_the_front_app(backend, + world): + world.frontmost_pid = 501 + world.windows = [window_info(1, pid=99), window_info(2, pid=501), + window_info(3, pid=501)] + assert backend.foreground_window() == 2, "front-to-back, so the first" + + +def test_no_frontmost_application_reads_as_no_window(backend, world): + world.frontmost_pid = None + assert backend.foreground_window() == 0 + + +def test_a_frontmost_app_with_no_ordinary_window_reads_as_none(backend, + world): + world.frontmost_pid = 501 + world.windows = [window_info(1, pid=501, layer=25)] + assert backend.foreground_window() == 0 + + +# --- rectangles and ownership ------------------------------------------------- + +def test_the_rectangle_is_the_bounds_quartz_reports(backend, world): + world.windows = [window_info(7, bounds=(100, 200, 640, 480))] + assert backend.window_rect(7) == (100, 200, 740, 680) + + +def test_a_rectangle_for_a_window_that_is_gone_is_none(backend, world): + # The lookup walks the whole list before giving up, so an unrelated + # window in front of it is the ordinary case, not an edge one. + world.windows = [window_info(9, bounds=(0, 0, 1, 1))] + assert backend.window_rect(7) is None + + +def test_a_window_quartz_reports_without_bounds_has_no_rectangle(backend, + world): + world.windows = [window_info(7)] + assert backend.window_rect(7) is None + + +def test_the_owning_pid_comes_from_the_window_info(backend, world): + world.windows = [window_info(7, pid=501)] + assert backend.window_process_id(7) == 501 + + +def test_a_window_that_is_gone_has_no_owner(backend, world): + assert backend.window_process_id(7) == 0 + + +# --- matching a Quartz window to an accessibility element --------------------- + +def _mac_with_window(number=7, *, pid=501, name="Editor", + bounds=(10, 20, 300, 400), ax_windows=None, + running=True): + world = World(windows=[window_info(number, name=name, pid=pid, + bounds=bounds)], + running_pids={pid} if running else set()) + world.ax_windows = {pid: list(ax_windows or [])} + return world + + +def _ax_window(origin=None, title=None): + attributes = {} + if origin is not None: + attributes["AXPosition"] = ("point", objc_stub.AXPoint(*origin)) + if title is not None: + attributes["AXTitle"] = title + return AXElement(**attributes) + + +def test_the_element_is_matched_by_its_origin(monkeypatch, on_darwin): + wanted = _ax_window(origin=(10, 20)) + world = _mac_with_window(ax_windows=[_ax_window(origin=(99, 99)), wanted]) + backend = _build(monkeypatch, world) + assert backend._ax_window(7) is wanted + + +def test_the_title_is_only_the_fallback(monkeypatch, on_darwin): + # Two windows cannot share an origin at one moment; they share titles all + # the time, so the frame is tried first and the title only after. + by_origin = _ax_window(origin=(10, 20), title="Other") + by_title = _ax_window(origin=(0, 0), title="Editor") + world = _mac_with_window(ax_windows=[by_title, by_origin]) + backend = _build(monkeypatch, world) + assert backend._ax_window(7) is by_origin + + +def test_a_title_match_is_used_when_no_origin_matches(monkeypatch, on_darwin): + by_title = _ax_window(origin=(0, 0), title="Editor") + world = _mac_with_window(ax_windows=[by_title]) + backend = _build(monkeypatch, world) + assert backend._ax_window(7) is by_title + + +def test_an_untitled_quartz_window_is_not_matched_by_an_empty_title( + monkeypatch, on_darwin): + # Matching "" against "" would pick an arbitrary window of the app. + candidate = _ax_window(origin=(0, 0), title="") + world = _mac_with_window(name="", ax_windows=[candidate]) + backend = _build(monkeypatch, world) + assert backend._ax_window(7) is None + + +def test_an_element_with_no_readable_position_is_skipped(monkeypatch, + on_darwin): + unreadable = _ax_window(title="Editor") + world = _mac_with_window(ax_windows=[unreadable]) + backend = _build(monkeypatch, world) + assert backend._ax_window(7) is unreadable, "found by title instead" + + +def test_a_window_quartz_does_not_know_has_no_element(backend): + assert backend._ax_window(7) is None + + +def test_a_window_with_no_owner_has_no_element(monkeypatch, on_darwin): + world = World(windows=[window_info(7, pid=0)]) + backend = _build(monkeypatch, world) + assert backend._ax_window(7) is None + + +def test_an_application_that_refuses_to_list_windows_yields_nothing( + monkeypatch, on_darwin): + world = _mac_with_window(ax_windows=[_ax_window(origin=(10, 20))]) + world.ax_list_error = AX_FAILURE + backend = _build(monkeypatch, world) + assert backend._ax_window(7) is None + + +@pytest.mark.parametrize("value,expected", [ + (None, (-1, -1)), + (("size", objc_stub.AXSize(1, 2)), (-1, -1)), + (("point", objc_stub.AXPoint(3, 4)), (3, 4)), +]) +def test_an_ax_point_reads_as_a_pair_or_as_nothing(monkeypatch, on_darwin, + world, value, expected): + objc_stub.install(monkeypatch, world) + assert _point(value) == expected + + +# --- minimised state ---------------------------------------------------------- + +def test_a_window_the_element_says_is_minimised_is_minimised(monkeypatch, + on_darwin): + element = _ax_window(origin=(10, 20)) + element.attributes["AXMinimized"] = True + world = _mac_with_window(ax_windows=[element]) + backend = _build(monkeypatch, world) + assert backend.is_minimized(7) is True + + +def test_a_window_the_element_says_is_not_minimised_is_not(monkeypatch, + on_darwin): + element = _ax_window(origin=(10, 20)) + element.attributes["AXMinimized"] = False + world = _mac_with_window(ax_windows=[element]) + backend = _build(monkeypatch, world) + assert backend.is_minimized(7) is False + + +def test_a_window_that_is_not_on_screen_at_all_is_minimised(backend): + # A minimised window is absent from the on-screen list, so failing to + # find it is itself the answer rather than an error. + assert backend.is_minimized(7) is True + + +def test_a_window_on_screen_with_no_element_is_not_minimised(monkeypatch, + on_darwin): + world = _mac_with_window(ax_windows=[]) + backend = _build(monkeypatch, world) + assert backend.is_minimized(7) is False + + +# --- raising ------------------------------------------------------------------ + +def test_raising_activates_the_app_and_then_the_window(monkeypatch, + on_darwin): + element = _ax_window(origin=(10, 20)) + world = _mac_with_window(ax_windows=[element]) + backend = _build(monkeypatch, world) + backend.set_foreground(7) + assert world.activated == [(501, 2)], "ignoring other apps" + assert element.actions == ["AXRaise"], ( + "activating brings the app's front window forward, not this one" + ) + + +def test_raising_a_window_with_no_owner_is_refused(monkeypatch, on_darwin): + world = World(windows=[window_info(7, pid=0)]) + backend = _build(monkeypatch, world) + with pytest.raises(AutoControlUnsupportedOperationException, + match="set_foreground"): + backend.set_foreground(7) + + +def test_raising_still_raises_when_the_app_object_is_gone(monkeypatch, + on_darwin): + # The application can quit between the pid lookup and the activation, + # which leaves nothing to activate but still an element to raise. + element = _ax_window(origin=(10, 20)) + world = _mac_with_window(ax_windows=[element], running=False) + backend = _build(monkeypatch, world) + backend.set_foreground(7) + assert world.activated == [] + assert element.actions == ["AXRaise"] + + +def test_raising_a_window_the_accessibility_api_cannot_find_still_activates( + monkeypatch, on_darwin): + # Without Accessibility there is no element, but the application can + # still be brought forward -- which is most of what the user asked for. + world = _mac_with_window(ax_windows=[]) + backend = _build(monkeypatch, world) + backend.set_foreground(7) + assert world.activated == [(501, 2)] + + +# --- restoring and show-state codes ------------------------------------------- + +def test_restoring_clears_the_minimised_attribute(monkeypatch, on_darwin): + element = _ax_window(origin=(10, 20)) + element.attributes["AXMinimized"] = True + world = _mac_with_window(ax_windows=[element]) + backend = _build(monkeypatch, world) + backend.restore(7) + assert element.attributes["AXMinimized"] is False + + +def test_restoring_a_window_with_no_element_says_why(monkeypatch, on_darwin): + # This is the TCC case: without Accessibility there is no element, and a + # silent no-op would look like a window that refused to restore. + world = _mac_with_window(ax_windows=[]) + backend = _build(monkeypatch, world) + with pytest.raises(AutoControlUnsupportedOperationException, + match="Accessibility"): + backend.restore(7) + + +@pytest.mark.parametrize("code", [macos.SW_SHOWNORMAL, macos.SW_RESTORE]) +def test_the_normal_and_restore_codes_both_restore(monkeypatch, on_darwin, + code): + element = _ax_window(origin=(10, 20)) + element.attributes["AXMinimized"] = True + world = _mac_with_window(ax_windows=[element]) + backend = _build(monkeypatch, world) + backend.show(7, code) + assert element.attributes["AXMinimized"] is False + + +@pytest.mark.parametrize("code", [macos.SW_MINIMIZE, macos.SW_SHOWMINIMIZED]) +def test_both_minimise_codes_minimise(monkeypatch, on_darwin, code): + element = _ax_window(origin=(10, 20)) + world = _mac_with_window(ax_windows=[element]) + backend = _build(monkeypatch, world) + backend.show(7, code) + assert element.attributes["AXMinimized"] is True + + +@pytest.mark.parametrize("code", [0, 3, 8]) +def test_a_show_code_with_no_macos_meaning_is_refused(backend, code): + # macOS has no hide-this-window and no maximise-this-window; zoom is not + # maximise, and the difference matters to the caller. + with pytest.raises(AutoControlUnsupportedOperationException, + match="show"): + backend.show(7, code) + + +# --- closing and minimising --------------------------------------------------- + +def test_closing_presses_the_windows_own_close_button(monkeypatch, on_darwin): + button = AXElement() + element = _ax_window(origin=(10, 20)) + element.attributes["AXCloseButton"] = button + world = _mac_with_window(ax_windows=[element]) + backend = _build(monkeypatch, world) + assert backend.close(7) is True + assert button.actions == ["AXPress"] + + +def test_a_close_the_accessibility_api_refuses_reports_failure(monkeypatch, + on_darwin): + # AX answers with an error code where zero is success, so this is the + # test that would fail if that were read the other way round. + button = AXElement() + button.action_error = AX_FAILURE + element = _ax_window(origin=(10, 20)) + element.attributes["AXCloseButton"] = button + world = _mac_with_window(ax_windows=[element]) + backend = _build(monkeypatch, world) + assert backend.close(7) is False + + +def test_a_window_with_no_close_button_cannot_be_closed(monkeypatch, + on_darwin): + world = _mac_with_window(ax_windows=[_ax_window(origin=(10, 20))]) + backend = _build(monkeypatch, world) + assert backend.close(7) is False + + +def test_closing_a_window_with_no_element_reports_failure(backend): + assert backend.close(7) is False + + +def test_minimising_sets_the_attribute(monkeypatch, on_darwin): + element = _ax_window(origin=(10, 20)) + world = _mac_with_window(ax_windows=[element]) + backend = _build(monkeypatch, world) + assert backend.minimize(7) is True + assert element.attributes["AXMinimized"] is True + + +def test_a_minimise_the_accessibility_api_refuses_reports_failure(monkeypatch, + on_darwin): + element = _ax_window(origin=(10, 20)) + element.set_error = AX_FAILURE + world = _mac_with_window(ax_windows=[element]) + backend = _build(monkeypatch, world) + assert backend.minimize(7) is False + + +def test_minimising_a_window_with_no_element_reports_failure(backend): + assert backend.minimize(7) is False + + +# --- moving ------------------------------------------------------------------- + +def test_moving_sets_position_and_size_as_ax_values(monkeypatch, on_darwin): + element = _ax_window(origin=(10, 20)) + world = _mac_with_window(ax_windows=[element]) + backend = _build(monkeypatch, world) + assert backend.move(7, 100, 200, 640, 480) is True + written = dict(element.assignments) + kind, point = written["AXPosition"] + assert (kind, point.x, point.y) == ("point", 100.0, 200.0) + kind, size = written["AXSize"] + assert (kind, size.width, size.height) == ("size", 640.0, 480.0) + + +def test_a_move_the_accessibility_api_refuses_reports_failure(monkeypatch, + on_darwin): + element = _ax_window(origin=(10, 20)) + element.set_error = AX_FAILURE + world = _mac_with_window(ax_windows=[element]) + backend = _build(monkeypatch, world) + assert backend.move(7, 0, 0, 10, 10) is False + + +def test_moving_a_window_with_no_element_reports_failure(backend): + assert backend.move(7, 0, 0, 10, 10) is False + + +# --- what macOS deliberately will not do -------------------------------------- + +@pytest.mark.parametrize("call", [ + lambda b: b.post_key(7, 38), + lambda b: b.post_click(7, "left", 0, 0), +]) +def test_input_to_an_unfocused_window_is_refused_not_faked(backend, call): + # macOS has no PostMessage and no XSendEvent: an event goes wherever the + # focus is. A "success" that clicked somewhere else is worse than a no. + with pytest.raises(AutoControlUnsupportedOperationException): + call(backend) diff --git a/test/unit_test/headless/test_window_backend_selection.py b/test/unit_test/headless/test_window_backend_selection.py new file mode 100644 index 00000000..dc18cb7d --- /dev/null +++ b/test/unit_test/headless/test_window_backend_selection.py @@ -0,0 +1,299 @@ +"""Which window backend a platform gets, and what the others refuse. + +Window management was Windows-only for the project's whole life -- the facade +branched on `sys.platform` and raised `NotImplementedError` everywhere else, +leaving 23 `AC_*` commands dead on macOS and Linux. This package is the seam +that replaced that branch, and the seam itself was covered by nothing: the +selector only ever ran its own platform's arm, and every square skipped the +other two. + +Three things here are worth stating in a test: + +* **A platform without a working backend gets a refusal that says why.** Not + an ImportError at start-up and not a backend that answers falsely -- a null + backend carrying the reason, so `AC_list_window` on a Wayland session says + "Wayland does not expose other windows to a client" rather than failing + with a stack trace about Xlib. +* **Selection is cached, because probing costs a connection.** It opens an X + display or runs a Quartz query, and the answer cannot change inside a + process. +* **"Cannot" and "did not work this time" are different answers.** The base + class raises for the first and every backend returns a falsy value for the + second; a caller that cannot tell them apart retries forever. + +Both stubbed platforms are driven from every square, so this file tests all +three arms wherever it runs -- which is the point, since the coverage floor +is the lowest square. +""" +from __future__ import annotations + +import sys +import types + +import pytest + +from headless import _pyobjc_stub as objc_stub +from headless import _xlib_stub +from je_auto_control.utils.exception.exceptions import ( + AutoControlUnsupportedOperationException, +) +from je_auto_control.wrapper import window_backends +from je_auto_control.wrapper.window_backends import ( + NullWindowBackend, WindowManageBackend, get_backend, reset_backend_cache, +) +from je_auto_control.wrapper.window_backends.windows_backend import ( + WindowsWindowBackend, +) + + +@pytest.fixture(autouse=True) +def clean_cache(): + """The selector caches; a test that leaves one behind poisons the next.""" + reset_backend_cache() + yield + reset_backend_cache() + + +@pytest.fixture +def on_platform(monkeypatch): + def _set(name: str): + monkeypatch.setattr(window_backends.sys, "platform", name) + return _set + + +# --- selection ---------------------------------------------------------------- + +@pytest.mark.parametrize("platform", ["win32", "cygwin", "msys"]) +def test_every_windows_spelling_selects_the_win32_backend(on_platform, + platform): + on_platform(platform) + assert isinstance(get_backend(), WindowsWindowBackend) + + +def test_a_mac_with_pyobjc_selects_the_quartz_backend(on_platform, monkeypatch): + on_platform("darwin") + objc_stub.install(monkeypatch, objc_stub.World()) + assert get_backend().name == "macos-quartz-ax" + + +def test_a_mac_without_pyobjc_gets_a_refusal_naming_it(on_platform, + monkeypatch): + on_platform("darwin") + objc_stub.install_missing(monkeypatch, "Quartz") + backend = get_backend() + assert isinstance(backend, NullWindowBackend) + assert "pyobjc" in backend.reason + + +@pytest.mark.parametrize("platform", ["linux", "linux2"]) +def test_a_linux_session_with_x_selects_the_ewmh_backend(on_platform, + monkeypatch, + platform): + on_platform(platform) + _xlib_stub.install(monkeypatch) + assert get_backend().name == "x11-ewmh" + + +def test_a_session_with_no_x_display_gets_a_refusal_naming_wayland( + on_platform, monkeypatch): + # Wayland deliberately does not let a client enumerate or move other + # applications' windows. That is a protocol decision, so the message has + # to point at XWayland rather than read as a missing dependency. + on_platform("linux") + _xlib_stub.install_failing(monkeypatch, RuntimeError("no DISPLAY")) + backend = get_backend() + assert isinstance(backend, NullWindowBackend) + assert "Wayland" in backend.reason + assert "XWayland" in backend.reason + + +def test_an_unknown_platform_gets_a_refusal_naming_it(on_platform): + on_platform("sunos5") + backend = get_backend() + assert isinstance(backend, NullWindowBackend) + assert "sunos5" in backend.reason + + +def test_the_choice_is_made_once_and_cached(on_platform): + # Probing opens an X connection or runs a Quartz query; the answer cannot + # change inside a process, so paying for it twice is pure cost. + on_platform("win32") + assert get_backend() is get_backend() + + +def test_resetting_the_cache_re_detects(on_platform): + on_platform("win32") + first = get_backend() + reset_backend_cache() + assert get_backend() is not first + + +# --- the null backend --------------------------------------------------------- + +def test_the_null_backend_lists_nothing_rather_than_refusing(): + # "There are no windows I can see" is a truthful answer a caller can + # iterate; raising here would make every listing a special case. + assert NullWindowBackend().list_windows() == [] + + +def test_the_null_backend_carries_its_reason_in_its_name(): + backend = NullWindowBackend("no X display") + assert backend.reason == "no X display" + assert "no X display" in backend.name + assert backend.available is False + + +def test_a_null_backend_with_no_reason_still_says_something(): + assert NullWindowBackend().reason + + +@pytest.mark.parametrize("call", [ + lambda b: b.foreground_window(), + lambda b: b.window_rect(1), + lambda b: b.window_process_id(1), + lambda b: b.is_minimized(1), + lambda b: b.set_foreground(1), + lambda b: b.restore(1), + lambda b: b.show(1, 9), + lambda b: b.close(1), + lambda b: b.minimize(1), + lambda b: b.move(1, 0, 0, 1, 1), + lambda b: b.post_key(1, 38), + lambda b: b.post_click(1, "left", 0, 0), +]) +def test_every_action_on_a_null_backend_refuses_loudly(call): + with pytest.raises(AutoControlUnsupportedOperationException): + call(NullWindowBackend("no X display")) + + +# --- the abstract base -------------------------------------------------------- + +def test_the_base_class_has_no_listing_of_its_own(): + # Every backend must answer this one; there is no sensible default, so + # the base leaves it abstract rather than returning an empty list. + with pytest.raises(NotImplementedError): + WindowManageBackend().list_windows() + + +def test_a_refusal_names_the_operation_and_the_backend(): + backend = WindowManageBackend() + with pytest.raises(AutoControlUnsupportedOperationException) as caught: + backend.restore(1) + assert "restore" in str(caught.value) + assert "abstract" in str(caught.value) + + +# --- the Windows backend's delegation ------------------------------------------ + +class _WinManager(types.ModuleType): + """The Win32 module, which stays the single home of the real calls.""" + + SW_RESTORE = 9 + + def __init__(self) -> None: + super().__init__("je_auto_control.windows.window.windows_window_manage") + self.calls = [] + + def _record(self, name, *args): + self.calls.append((name, args)) + return f"{name}-result" + + def get_all_window_hwnd(self): + return self._record("get_all_window_hwnd") + + def get_foreground_window(self): + return self._record("get_foreground_window") + + def get_window_rect(self, window_id): + return self._record("get_window_rect", window_id) + + def get_window_process_id(self, window_id): + return self._record("get_window_process_id", window_id) + + def is_window_minimized(self, window_id): + return self._record("is_window_minimized", window_id) + + def set_foreground_window(self, window_id): + return self._record("set_foreground_window", window_id) + + def show_window(self, window_id, cmd_show): + return self._record("show_window", window_id, cmd_show) + + def close_window(self, window_id): + return self._record("close_window", window_id) + + def minimize_window(self, window_id): + return self._record("minimize_window", window_id) + + def move_window(self, window_id, x, y, width, height): + return self._record("move_window", window_id, x, y, width, height) + + def post_key(self, window_id, keycode, character): + return self._record("post_key", window_id, keycode, character) + + def post_click(self, window_id, button, x, y): + return self._record("post_click", window_id, button, x, y) + + +@pytest.fixture +def win_manager(monkeypatch): + """Stand in for the Win32 module, on every platform including Windows. + + The backend reaches it with `from je_auto_control.windows.window import + windows_window_manage`, and `from package import name` resolves by + *attribute* on the package before it looks in `sys.modules` -- so putting + the double under its own dotted name is not enough on a machine where the + real one is importable. The package it is read off is replaced instead. + """ + module = _WinManager() + package = types.ModuleType("je_auto_control.windows.window") + package.windows_window_manage = module + monkeypatch.setitem(sys.modules, "je_auto_control.windows.window", package) + monkeypatch.setitem( + sys.modules, "je_auto_control.windows.window.windows_window_manage", + module) + return module + + +@pytest.mark.parametrize("call,expected", [ + (lambda b: b.list_windows(), ("get_all_window_hwnd", ())), + (lambda b: b.foreground_window(), ("get_foreground_window", ())), + (lambda b: b.window_rect(5), ("get_window_rect", (5,))), + (lambda b: b.window_process_id(5), ("get_window_process_id", (5,))), + (lambda b: b.is_minimized(5), ("is_window_minimized", (5,))), + (lambda b: b.set_foreground(5), ("set_foreground_window", (5,))), + (lambda b: b.show(5, 3), ("show_window", (5, 3))), + (lambda b: b.close(5), ("close_window", (5,))), + (lambda b: b.minimize(5), ("minimize_window", (5,))), + (lambda b: b.move(5, 1, 2, 3, 4), ("move_window", (5, 1, 2, 3, 4))), + (lambda b: b.post_key(5, 38, "a"), ("post_key", (5, 38, "a"))), + (lambda b: b.post_click(5, "left", 1, 2), ("post_click", + (5, "left", 1, 2))), +]) +def test_the_windows_backend_forwards_to_the_win32_module(win_manager, call, + expected): + call(WindowsWindowBackend()) + assert win_manager.calls == [expected] + + +def test_restoring_goes_through_the_win32_show_state_constant(win_manager): + # `restore` is deliberately not "show it however": it un-minimises + # without un-maximising, which is what SW_RESTORE means. + WindowsWindowBackend().restore(5) + assert win_manager.calls == [("show_window", (5, _WinManager.SW_RESTORE))] + + +def test_a_show_code_is_passed_through_as_an_int(win_manager): + WindowsWindowBackend().show(5, "3") + assert win_manager.calls == [("show_window", (5, 3))] + + +@pytest.mark.parametrize("platform,available", [ + ("win32", True), ("cygwin", True), ("msys", True), + ("linux", False), ("darwin", False), +]) +def test_the_windows_backend_knows_where_it_can_run(monkeypatch, platform, + available): + from je_auto_control.wrapper.window_backends import windows_backend + monkeypatch.setattr(windows_backend.sys, "platform", platform) + assert WindowsWindowBackend().available is available diff --git a/test/unit_test/headless/test_window_backend_x11.py b/test/unit_test/headless/test_window_backend_x11.py new file mode 100644 index 00000000..3c4fa23a --- /dev/null +++ b/test/unit_test/headless/test_window_backend_x11.py @@ -0,0 +1,553 @@ +"""Window management through EWMH, from any square rather than only Linux. + +`x11_backend.py` was at 0% everywhere. `Progress.md` had it down as "only the +two Linux squares can run it" -- but every `Xlib` import in the module is +*inside* a method, so the module loads on Windows and macOS too and only the +calls need a display. A stub in `sys.modules` (`_xlib_stub.py`) reaches all of +it from every square, which is what makes it worth writing: the coverage floor +is the lowest square, so a test that runs on two of nine moves it least. + +What is actually being checked is protocol arithmetic, and it is the kind +that fails silently: + +* **EWMH requests go to the *root*, not to the window.** They are addressed + there with `SubstructureRedirect`, which is what routes them to the window + manager -- the only party allowed to act on them. Sending to the window + instead is a message nobody answers. +* **Stacking order is bottom-to-top.** `list_windows` reverses it so the + front-most window is first, which is what makes "the first title that + matches" the one the user meant. `_NET_CLIENT_LIST` carries no order at + all, so it is only a fallback and must *not* be reversed. +* **`move()` sizes the client while `window_rect()` reads the frame.** EWMH + and Win32 disagree about which rectangle they mean, so the decorations + come off the requested size; getting that wrong is a window that shrinks a + title bar's worth every time a script round-trips it. +* **Source indication 2 means "pager".** A window manager honours it without + the focus-stealing prevention it applies to source 1, so a raise sent as an + application is a raise that quietly does not happen. + +The stub's constants carry their real `X.h` values, and +`test_xlib_stub_values.py` holds them to the installed library wherever there +is one -- so the masks asserted below are the numbers that go on the wire. +""" +from __future__ import annotations + +import pytest + +from headless import _xlib_stub +from je_auto_control.utils.exception.exceptions import ( + AutoControlUnsupportedOperationException, +) +from je_auto_control.wrapper.window_backends import x11_backend as x11 +from je_auto_control.wrapper.window_backends.x11_backend import ( + X11WindowBackend, _as_text, +) + +_X = _xlib_stub.X_CONSTANTS +_REDIRECT = (_X["SubstructureRedirectMask"] | _X["SubstructureNotifyMask"]) + + +@pytest.fixture +def on_linux(monkeypatch): + """Answer `_is_linux()` the way an X session would.""" + monkeypatch.setattr(x11.sys, "platform", "linux") + + +@pytest.fixture +def display(monkeypatch, on_linux): + return _xlib_stub.install(monkeypatch) + + +@pytest.fixture +def backend(display): + instance = X11WindowBackend() + assert instance.available, "the stub display opens" + return instance + + +def _client_messages(display): + """Every EWMH request the backend addressed to the root window.""" + return [(event.fields["client_type"], event.fields["data"], mask) + for event, mask, _propagate in display.root.sent] + + +def _named_messages(display): + """The same, with atoms resolved back to the names they were interned as.""" + return [(display.atom_name(atom), data[1], mask) + for atom, data, mask in _client_messages(display)] + + +# --- availability ------------------------------------------------------------- + +def test_the_backend_is_unavailable_off_linux(monkeypatch): + monkeypatch.setattr(x11.sys, "platform", "win32") + _xlib_stub.install(monkeypatch) + backend = X11WindowBackend() + assert backend.available is False + assert backend._connection is None, "it did not even try to connect" + + +@pytest.mark.parametrize("platform", ["linux", "linux2"]) +def test_both_spellings_of_linux_are_linux(monkeypatch, platform): + monkeypatch.setattr(x11.sys, "platform", platform) + _xlib_stub.install(monkeypatch) + assert X11WindowBackend().available is True + + +def test_a_session_with_no_display_is_unavailable_rather_than_fatal( + monkeypatch, on_linux): + # No `DISPLAY` is the ordinary state of a Wayland session or an ssh + # login; the selector turns this into a null backend with a reason. + _xlib_stub.install_failing(monkeypatch, RuntimeError("no DISPLAY")) + assert X11WindowBackend().available is False + + +def test_the_backend_names_the_protocol_it_speaks(backend): + assert backend.name == "x11-ewmh" + + +# --- the connection and its atoms --------------------------------------------- + +def test_the_display_is_opened_once_and_kept(backend, display): + assert backend._display() is display + assert backend._display() is display + + +def test_an_atom_is_interned_once_per_connection(backend, display): + first = backend._atom("_NET_WM_STATE") + assert backend._atom("_NET_WM_STATE") == first + assert list(display.atoms) == ["_NET_WM_STATE"] + + +def test_a_missing_property_reads_as_none(backend, display): + assert backend._property(display.root, "_NET_CLIENT_LIST") is None + + +# --- listing ------------------------------------------------------------------ + +def test_listing_puts_the_front_most_window_first(backend, display): + # `_NET_CLIENT_LIST_STACKING` is bottom-to-top, so 30 is in front. + display.set_root_property("_NET_CLIENT_LIST_STACKING", [10, 20, 30]) + for window_id, title in ((10, "back"), (20, "middle"), (30, "front")): + _titled(display, window_id, title) + assert backend.list_windows() == [(30, "front"), (20, "middle"), + (10, "back")] + + +def test_listing_falls_back_to_the_unordered_list(backend, display): + # `_NET_CLIENT_LIST` has no stacking meaning, so reversing it would + # invent an order the window manager never stated. + display.set_root_property("_NET_CLIENT_LIST", [10, 20]) + _titled(display, 10, "first") + _titled(display, 20, "second") + assert backend.list_windows() == [(10, "first"), (20, "second")] + + +def test_a_desktop_with_no_windows_lists_nothing(backend, display): + assert backend.list_windows() == [] + + +def test_a_title_prefers_the_utf8_property(backend, display): + display.set_root_property("_NET_CLIENT_LIST_STACKING", [10]) + display.intern_atom("_NET_WM_NAME") + display.window(10, properties={"_NET_WM_NAME": b"Editor \xe2\x80\x94 file"}, + wm_name="legacy") + assert backend.list_windows() == [(10, "Editor — file")] + + +def test_a_title_falls_back_to_the_legacy_property(backend, display): + display.set_root_property("_NET_CLIENT_LIST_STACKING", [10]) + display.intern_atom("_NET_WM_NAME") + display.window(10, wm_name="Old Toolkit") + assert backend.list_windows() == [(10, "Old Toolkit")] + + +def test_a_window_that_vanishes_mid_walk_lists_with_an_empty_title( + backend, display): + # The list is a snapshot; by the time a title is read the window may be + # gone, and one closing window must not fail the whole enumeration. + display.set_root_property("_NET_CLIENT_LIST_STACKING", [10, 20]) + _titled(display, 10, "still here") + display.window(20).property_error = RuntimeError("BadWindow") + assert backend.list_windows() == [(20, ""), (10, "still here")] + + +def _titled(display, window_id: int, title: str): + display.intern_atom("_NET_WM_NAME") + return display.window(window_id, + properties={"_NET_WM_NAME": title.encode("utf-8")}) + + +# --- the focused window ------------------------------------------------------- + +def test_the_foreground_window_is_the_first_id_the_property_names(backend, + display): + display.set_root_property("_NET_ACTIVE_WINDOW", [42]) + assert backend.foreground_window() == 42 + + +@pytest.mark.parametrize("value", [None, []]) +def test_no_focused_window_reads_as_zero(backend, display, value): + if value is not None: + display.set_root_property("_NET_ACTIVE_WINDOW", value) + assert backend.foreground_window() == 0 + + +# --- rectangles --------------------------------------------------------------- + +def test_the_rectangle_is_the_frame_not_the_client(backend, display): + # A reparenting window manager makes the client a grandchild of the root + # inside a frame that carries the border and title bar; the frame is what + # the user sees and drags, and what Win32's GetWindowRect would report. + display.window(50, parent_id=99, geometry=(0, 0, 10, 10)) + display.window(99, parent_id=1, geometry=(100, 200, 640, 480)) + assert backend.window_rect(50) == (100, 200, 740, 680) + + +def test_an_undecorated_window_is_its_own_frame(backend, display): + display.window(50, parent_id=1, geometry=(5, 6, 70, 80)) + assert backend.window_rect(50) == (5, 6, 75, 86) + + +def test_a_window_whose_parent_is_gone_is_its_own_frame(backend, display): + display.window(50, parent_id=None, geometry=(1, 2, 3, 4)) + assert backend.window_rect(50) == (1, 2, 4, 6) + + +def test_a_frame_walk_cannot_loop_forever(backend, display): + # A cycle in the tree would hang the caller; the walk is bounded instead. + display.window(50, parent_id=51, geometry=(0, 0, 1, 1)) + display.window(51, parent_id=50, geometry=(7, 7, 2, 2)) + assert backend.window_rect(50) is not None + + +def test_a_rectangle_for_a_window_that_is_gone_is_none(backend, display): + display.window(50, parent_id=1) + display.window(50).geometry_error = RuntimeError("BadWindow") + assert backend.window_rect(50) is None + + +# --- ownership and state ------------------------------------------------------ + +def test_the_owning_pid_comes_from_the_ewmh_property(backend, display): + display.intern_atom("_NET_WM_PID") + display.window(50, properties={"_NET_WM_PID": [4321]}) + assert backend.window_process_id(50) == 4321 + + +def test_a_window_with_no_pid_property_reports_zero(backend, display): + assert backend.window_process_id(50) == 0 + + +def test_a_pid_read_that_fails_reports_zero(backend, display): + display.window(50).property_error = RuntimeError("BadWindow") + assert backend.window_process_id(50) == 0 + + +def test_a_hidden_window_is_minimised(backend, display): + hidden = backend._atom("_NET_WM_STATE_HIDDEN") + display.intern_atom("_NET_WM_STATE") + display.window(50, properties={"_NET_WM_STATE": [hidden]}) + assert backend.is_minimized(50) is True + + +def test_a_window_in_another_state_is_not_minimised(backend, display): + display.intern_atom("_NET_WM_STATE") + other = display.intern_atom("_NET_WM_STATE_MAXIMIZED_VERT") + display.window(50, properties={"_NET_WM_STATE": [other]}) + assert backend.is_minimized(50) is False + + +def test_a_window_with_no_state_property_is_not_minimised(backend, display): + assert backend.is_minimized(50) is False + + +def test_a_state_read_that_fails_is_not_minimised(backend, display): + display.window(50).property_error = RuntimeError("BadWindow") + assert backend.is_minimized(50) is False + + +# --- raising and restoring ---------------------------------------------------- + +def test_raising_a_window_addresses_the_root_as_a_pager(backend, display): + backend.set_foreground(50) + [(name, source, mask)] = _named_messages(display) + assert name == "_NET_ACTIVE_WINDOW" + assert source[0] == 2, "source 2 is 'pager'; source 1 gets refused" + assert mask == _REDIRECT + assert display.root.sent[0][0].fields["window"].id == 50 + + +def test_the_request_carries_the_window_it_is_about(backend, display): + backend.set_foreground(50) + event = display.root.sent[0][0] + assert event.fields["window"].id == 50, "addressed to root, about 50" + + +def test_a_client_message_is_padded_to_five_values(backend, display): + backend.set_foreground(50) + _format, data = display.root.sent[0][0].fields["data"] + assert _format == 32 + assert len(data) == 5 + + +def test_restoring_maps_the_window_before_raising_it(backend, display): + backend.restore(50) + assert display.window(50).mapped is True + assert [name for name, _, _ in _named_messages(display)] == [ + "_NET_ACTIVE_WINDOW", + ] + + +# --- show-state codes --------------------------------------------------------- + +def test_hiding_unmaps_the_window(backend, display): + backend.show(50, x11.SW_HIDE) + assert display.window(50).mapped is False + + +@pytest.mark.parametrize("code", [x11.SW_SHOWNORMAL, x11.SW_RESTORE]) +def test_the_normal_and_restore_codes_both_restore(backend, display, code): + backend.show(50, code) + assert display.window(50).mapped is True + + +@pytest.mark.parametrize("code", [x11.SW_MINIMIZE, x11.SW_SHOWMINIMIZED]) +def test_both_minimise_codes_send_the_icccm_request(backend, display, code): + backend.show(50, code) + assert [name for name, _, _ in _named_messages(display)] == [ + "WM_CHANGE_STATE", + ] + + +def test_maximising_names_both_axes(backend, display): + # `_NET_WM_STATE_ADD` is 1, and a request that names one axis grows the + # window one way only. + backend.show(50, x11.SW_MAXIMIZE) + [(name, data, _mask)] = _named_messages(display) + assert name == "_NET_WM_STATE" + assert data[0] == 1, "_NET_WM_STATE_ADD" + assert {data[1], data[2]} == { + backend._atom("_NET_WM_STATE_MAXIMIZED_HORZ"), + backend._atom("_NET_WM_STATE_MAXIMIZED_VERT"), + } + + +def test_a_show_code_with_no_x11_meaning_is_refused(backend, display): + # SW_SHOWNA, SW_FORCEMINIMIZE and friends have no X11 equivalent, and + # guessing at one would move a window the caller did not ask to move. + with pytest.raises(AutoControlUnsupportedOperationException, + match="show"): + backend.show(50, 8) + assert display.root.sent == [] + + +# --- closing and minimising --------------------------------------------------- + +def test_closing_sends_the_ewmh_close_request(backend, display): + assert backend.close(50) is True + [(name, data, _mask)] = _named_messages(display) + assert name == "_NET_CLOSE_WINDOW" + assert data[1] == 2, "source 2, as with every other request here" + + +def test_a_close_that_the_connection_refuses_reports_failure(backend, display): + display.root.send_error = RuntimeError("connection lost") + assert backend.close(50) is False + + +def test_minimising_uses_the_icccm_request(backend, display): + # There is no `_NET_` message for iconify; `WM_CHANGE_STATE` with the + # ICCCM iconic state is what every window manager implements. + assert backend.minimize(50) is True + [(name, data, _mask)] = _named_messages(display) + assert name == "WM_CHANGE_STATE" + assert data[0] == 3, "IconicState" + + +def test_a_minimise_that_the_connection_refuses_reports_failure(backend, + display): + display.root.send_error = RuntimeError("connection lost") + assert backend.minimize(50) is False + + +# --- moving ------------------------------------------------------------------- + +def test_moving_takes_the_decorations_off_the_requested_size(backend, display): + # EWMH sizes the client; Win32's MoveWindow sizes the frame. Subtracting + # the extents is what makes move() round-trip with window_rect(). + display.intern_atom("_NET_FRAME_EXTENTS") + display.window(50, properties={"_NET_FRAME_EXTENTS": [4, 4, 30, 2]}) + assert backend.move(50, 100, 200, 640, 480) is True + [(name, data, _mask)] = _named_messages(display) + assert name == "_NET_MOVERESIZE_WINDOW" + assert data[1:] == [100, 200, 640 - 8, 480 - 32] + + +def test_moving_a_window_with_no_extents_uses_the_size_as_given(backend, + display): + backend.move(50, 10, 20, 300, 400) + [(_name, data, _mask)] = _named_messages(display) + assert data[1:] == [10, 20, 300, 400] + + +def test_an_unreadable_extents_property_is_treated_as_no_decorations(backend, + display): + # Not knowing the border thickness is not a reason to refuse the move: + # the window ends up a border's worth larger, which is recoverable, while + # refusing leaves the caller with a window that never moved at all. + display.window(50).property_error = RuntimeError("BadWindow") + assert backend.move(50, 0, 0, 100, 100) is True + [(_name, data, _mask)] = _named_messages(display) + assert data[3:] == [100, 100] + + +def test_a_short_extents_property_is_treated_as_no_decorations(backend, + display): + display.intern_atom("_NET_FRAME_EXTENTS") + display.window(50, properties={"_NET_FRAME_EXTENTS": [4, 4]}) + backend.move(50, 0, 0, 100, 100) + [(_name, data, _mask)] = _named_messages(display) + assert data[3:] == [100, 100] + + +def test_a_window_smaller_than_its_own_decorations_still_gets_a_size(backend, + display): + # A client size of zero or less is not a request any window manager can + # honour, so it floors at one pixel rather than going negative. + display.intern_atom("_NET_FRAME_EXTENTS") + display.window(50, properties={"_NET_FRAME_EXTENTS": [40, 40, 40, 40]}) + backend.move(50, 0, 0, 10, 10) + [(_name, data, _mask)] = _named_messages(display) + assert data[3:] == [1, 1] + + +def test_the_move_flags_name_all_four_fields_and_the_pager_source(backend, + display): + backend.move(50, 0, 0, 100, 100) + [(_name, data, _mask)] = _named_messages(display) + flags = data[0] + for bit in (8, 9, 10, 11): + assert flags & (1 << bit), f"bit {bit} says one of x/y/w/h is supplied" + assert (flags >> 12) & 0b11 == 2, "source indication 2 is 'pager'" + + +def test_a_move_the_connection_refuses_reports_failure(backend, display): + display.root.send_error = RuntimeError("connection lost") + assert backend.move(50, 0, 0, 100, 100) is False + + +# --- posting input to an unfocused window ------------------------------------- + +def test_posting_a_key_sends_a_press_and_a_release(backend, display): + assert backend.post_key(50, keycode=38, character="a") is True + sent = display.window(50).sent + assert [event.kind for event, _mask, _p in sent] == ["KeyPress", + "KeyRelease"] + assert [mask for _e, mask, _p in sent] == [_X["KeyPressMask"], + _X["KeyReleaseMask"]] + + +def test_a_posted_key_is_addressed_by_keycode_not_by_character(backend, + display): + # X11 has no "type this character" event; the character is Win32's idea + # and is dropped rather than guessed at. + backend.post_key(50, keycode=38, character="Z") + press = display.window(50).sent[0][0] + assert press.fields["detail"] == 38 + assert "character" not in press.fields + + +def test_a_posted_key_is_marked_as_propagating(backend, display): + # Synthetic events are what XSendEvent delivers; propagate lets the + # toolkit that does accept them see it on an ancestor. + backend.post_key(50, keycode=38) + assert display.window(50).sent[0][2] is True + + +def test_a_post_key_that_the_connection_refuses_reports_failure(backend, + display): + display.window(50).send_error = RuntimeError("connection lost") + assert backend.post_key(50, keycode=38) is False + + +@pytest.mark.parametrize("button,number", [ + ("left", 1), ("middle", 2), ("right", 3), + ("LEFT", 1), ("mouse_right", 3), +]) +def test_posting_a_click_maps_the_button_name_to_x11_numbering( + backend, display, button, number): + assert backend.post_click(50, button, x=7, y=9) is True + press = display.window(50).sent[0][0] + assert press.fields["detail"] == number + + +def test_a_posted_click_carries_window_relative_coordinates(backend, display): + backend.post_click(50, "left", x=7, y=9) + press = display.window(50).sent[0][0] + assert (press.fields["event_x"], press.fields["event_y"]) == (7, 9) + assert (press.fields["root_x"], press.fields["root_y"]) == (0, 0) + + +def test_posting_a_click_sends_a_press_and_a_release(backend, display): + backend.post_click(50, "left", x=0, y=0) + sent = display.window(50).sent + assert [event.kind for event, _mask, _p in sent] == ["ButtonPress", + "ButtonRelease"] + assert [mask for _e, mask, _p in sent] == [_X["ButtonPressMask"], + _X["ButtonReleaseMask"]] + + +def test_an_unknown_button_is_refused_rather_than_guessed(backend, display): + with pytest.raises(AutoControlUnsupportedOperationException, + match="post_click"): + backend.post_click(50, "thumb", x=0, y=0) + assert display.window(50).sent == [] + + +def test_a_post_click_that_the_connection_refuses_reports_failure(backend, + display): + display.window(50).send_error = RuntimeError("connection lost") + assert backend.post_click(50, "left", x=0, y=0) is False + + +# --- every request is flushed ------------------------------------------------- + +@pytest.mark.parametrize("action", [ + lambda b: b.set_foreground(50), + lambda b: b.restore(50), + lambda b: b.show(50, x11.SW_HIDE), + lambda b: b.close(50), + lambda b: b.minimize(50), + lambda b: b.move(50, 0, 0, 10, 10), + lambda b: b.post_key(50, 38), + lambda b: b.post_click(50, "left", 0, 0), +]) +def test_every_action_flushes_the_connection(backend, display, action): + # X buffers requests; one that is never flushed is one the window manager + # does not see until something else happens to flush it. + before = display.flushes + action(backend) + assert display.flushes > before + + +# --- property decoding -------------------------------------------------------- + +@pytest.mark.parametrize("value,expected", [ + (None, ""), + (b"plain", "plain"), + ("already text", "already text"), + (b"caf\xc3\xa9", "café"), + ([104, 105], "hi"), + ([104, 105, 0, 0], "hi"), + (b"\xff\xfe", "��"), +]) +def test_a_property_value_decodes_to_text(value, expected): + # X properties arrive as bytes, as str, or as an array of byte-sized + # ints depending on the property and the Xlib version, and a title that + # is not valid UTF-8 is a title to show badly, not an exception. + assert _as_text(value) == expected + + +def test_an_undecodable_property_falls_back_to_its_repr(): + assert _as_text(object) == str(object) diff --git a/test/unit_test/headless/test_xlib_stub_values.py b/test/unit_test/headless/test_xlib_stub_values.py new file mode 100644 index 00000000..35e51ab5 --- /dev/null +++ b/test/unit_test/headless/test_xlib_stub_values.py @@ -0,0 +1,75 @@ +"""Hold the Xlib stub to the library it stands in for. + +`_xlib_stub.py` lets `test_window_backend_x11.py` and its accessibility +counterpart run on all nine CI squares rather than the two with an X server, +by shadowing `Xlib` in `sys.modules`. That buys portability at one cost: a +stub can agree with a test about a number that is wrong on the wire. + +This file is what stops that. `python-Xlib` is a real dependency on Linux and +BSD, so on both Linux squares the genuine module is installed and every +constant the stub declares is compared against it. Elsewhere there is nothing +to compare with and these skip -- which is the right shape for the check, +because a wrong constant is a Linux-only bug and Linux is where it is caught. + +If a test here fails, the stub is wrong, not the library: fix +`_xlib_stub.py`'s value and re-read whichever assertion in +`test_window_backend_x11.py` depended on it. +""" +from __future__ import annotations + +import pytest + +from headless import _xlib_stub + +# python-Xlib ships no wheel-less platforms: it is pinned in pyproject for +# Linux, FreeBSD, OpenBSD and NetBSD, and absent everywhere else. +xlib = pytest.importorskip( + "Xlib", reason="python-Xlib is a Linux/BSD-only dependency", +) + + +@pytest.mark.parametrize("name", sorted(_xlib_stub.X_CONSTANTS)) +def test_every_x_constant_matches_the_installed_library(name): + from Xlib import X + assert _xlib_stub.X_CONSTANTS[name] == getattr(X, name), ( + f"the stub's X.{name} is not what python-Xlib says it is" + ) + + +@pytest.mark.parametrize("name", sorted(_xlib_stub.XATOM_CONSTANTS)) +def test_every_predefined_atom_matches_the_installed_library(name): + from Xlib import Xatom + assert _xlib_stub.XATOM_CONSTANTS[name] == getattr(Xatom, name), ( + f"the stub's Xatom.{name} is not what python-Xlib says it is" + ) + + +@pytest.mark.parametrize("name", sorted(_xlib_stub.KEYSYMS)) +def test_every_keysym_matches_what_the_installed_library_resolves(name): + # `XK.string_to_keysym` is what turns "Page_Up" into the number a grab is + # registered against; a wrong one grabs the wrong key. + from Xlib import XK + assert _xlib_stub.KEYSYMS[name] == XK.string_to_keysym(name), ( + f"the stub's keysym for {name!r} is not what python-Xlib resolves" + ) + + +def test_an_unknown_key_name_resolves_to_zero(): + # Zero is how X says "no such keysym", and the hotkey backend reports it + # as an unsupported key rather than grabbing keycode 0. + from Xlib import XK + assert XK.string_to_keysym("not-a-key") == 0 + + +def test_the_event_classes_the_stub_fakes_all_exist(): + # A stub that answers for an event class the library does not have would + # let a backend build a message nothing can send. + from Xlib.protocol import event + for name in ("ClientMessage", "KeyPress", "KeyRelease", + "ButtonPress", "ButtonRelease"): + assert hasattr(event, name), f"Xlib.protocol.event.{name}" + + +def test_the_display_entry_point_the_stub_fakes_exists(): + from Xlib import display + assert hasattr(display, "Display")