Skip to content

chore(ci): qualify native accessibility against the installed package (#31) - #200

Merged
mberrys merged 3 commits into
devfrom
chore/issue-31-native-a11y-qualification
Oct 5, 2026
Merged

mberrys merged 3 commits into
devfrom
chore/issue-31-native-a11y-qualification

Conversation

@mberrys

@mberrys mberrys commented Oct 4, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Qualify native accessibility and the Widgets-free graph against the installed package (L03-05, #31). The MSI and AppImage workflows now run the product Quick accessibility harness against a staged copy of build/install — the tree a user runs and the tree the Widgets-free profile is verified against — instead of the developer build tree, and the native/software distinction is structural in the record, not prose.

Refs #31. Topic PR into dev; the issue stays open until the promotion to the default branch (github closing keywords only act on a merge into stable).

Where the qualification belongs, and why

The native claim cannot be made from a developer build tree. A build-tree run loads the build-tree LoopLibCore.dll/LoopLibQuick.dll and resolves Qt from LOOP_QT_ROOT; only the installed tree contains the deployed Qt closure that the verify-widgets-free-release-profile.py --install-dir scan actually blesses. The steps therefore stay exactly where they are in the job order — after cmake --install and before packaging — but now stage the installed tree and run the harness from inside that copy.

The harness binary is a qualification harness, not product surface: installing it would make verify-loop-surface.ps1 flag it as an unmanifested first-party artifact and would ship it in the MSI. So scripts/lib/loop-quick-a11y-staging.ps1 copies build/install to a fresh staging directory and drops the harness in that copy's usr/bin. The shipped install tree is never mutated.

What changed

  • scripts/run-product-quick-a11y-smoke.ps1 — -InstallTree stages and runs against the installed closure; -Platform selects the Qt platform (Windows uses windows, i.e. real d3d11, not the offscreen software fallback); a non-offscreen native run fails if it falls back to the software rasterizer; each run writes a machine-readable record with the source SHA, installed-tree path, executable digest and observed backend.
  • scripts/run-installed-quick-a11y-uia.ps1 (new) — wires the previously-unwired tools/ProductQuickAccessibilitySmoke/inspect-windows-uia.ps1 driver (added by L02-04 — Expose accessible operator semantics #25) to the installed tree. This is the only lane that can prove the native accessibility backend is active: QAccessible::isActive() stays false until a real accessibility client attaches, which an offscreen harness run can never demonstrate.
  • scripts/ci/verify_quick_accessibility_evidence.py (new) — fail-closed gate over the records. Chosen over adding a parallel evidence document: it tightens the verification the workflow already runs rather than introducing a second, reconcilable standard. A missing, skipped or failed native lane fails; a software-only record offered as the native claim fails because it carries a different claim and no OS-accessibility observation.
  • LinuxInstall.yml / WindowsInstall.yml — native + software qualification against the installed tree, Windows additionally drives UI Automation and requires it, then the gate runs. Native stays fatal on both platforms.
  • Tests: scripts/ci/test_verify_quick_accessibility_evidence.py (14 cases, incl. the software-in-native-slot case) and a new test_workflow_contracts.py case pinning the installed-tree wiring, the Windows-requires-native-accessibility asymmetry, and lane order.
  • Docs: QUICK_ACCESSIBILITY_CONTRACT.md, ACCESSIBILITY_BASELINE.md, PACKAGING_LICENSING.md, CI.md.

Evidence

Source SHA (base = first parent): 0f2f7599e3e17ce4e57bfdedfce85582c8f51115. Commit: f860d22f. Install-tree identity qualified locally: C:\.dev\repos\loop-build-31\install; harness executable_sha256=2a740b974ec3702b81ee619582ef5229e54b46de0077217727149da4a9219a89, operator fixture fixture_sha256=31cc0710a652e2586131c1d87297f7a5988e68d79442b1e1d0f5b82ec2b9a49d.

Exercised on this Windows host (cmake --install into C:/.dev/repos/loop-build-31/install, staged copy, Qt removed from PATH so only the installed closure resolves):

Lane Result
Installed-tree native (-Platform windows) PASS, graphics_api=d3d11, artifact.scope=installed-tree
Installed-tree software (QT_QUICK_BACKEND=software) PASS, graphics_api=software, stale_rejected=1 on the finding-navigation fixture
Installed-tree native accessibility (Windows UI Automation) PASS, 42 observations across six operator states, graphics_api=d3d11, native_accessibility_backend_active=true
Gate verify_quick_accessibility_evidence.py over the three records PASS
check-change.py --base origin/dev PASS (status: pass, modules [documentation], all global guards green)
check-architecture.py --base origin/dev architecture contracts ok

A regression check confirms the base-tree lanes still behave as callers expect: build-tree native and software both PASS under offscreen.

Honest finding: the previous "native" lane was not native. Under QT_QPA_PLATFORM=offscreen the smoke reported graphics_api=software on both backends, so the old native evidence could not have supported a native claim. The installed-tree lanes now use -Platform windows on Windows and assert a non-software backend.

Unavailable / not exercised:

  • Linux installed package (AppImage): this host cannot build or install it; the Linux workflow edit is inference, not execution.
  • Linux native accessibility backend (AT-SPI): unavailable on a headless runner, so Linux records quick-a11y-native-accessibility-unavailable.txt rather than leaving the lane absent-and-fine.
  • MSI/AppImage package digest: the package is produced after this qualification, so the record carries the install-tree identity and harness digest; the package digest remains in the existing inspect_package_dependencies.py evidence.json. No package was installed locally — the staged install tree was qualified.

Anti-slop review

The diff moves the accessibility qualification onto the installed tree and makes the native/software distinction structural, without restructuring any build contract. The accepted risk is that the Linux OS-accessibility lane is genuinely unavailable headless — recorded as unavailable rather than as a pass — while the Windows UI Automation lane was exercised locally (42 observations, d3d11) and is otherwise only proven by the hosted run. One duplicated harness locator remains between the smoke script and the staging lib; it is small and left for a later cleanup rather than widening this change.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

mberrys and others added 3 commits October 3, 2026 23:30
Run the product Quick accessibility qualification against a staged copy of
the installed tree, not the developer build tree, on Linux and Windows. The
native and software runs write separate records with distinct claims; Windows
drives the native UI Automation client and requires it; Linux records the OS
accessibility lane unavailable. scripts/ci/verify_quick_accessibility_evidence.py
fails closed when a required lane is missing, failed, captured outside an
installed tree, or replaced by a software-only record.
…lane on any platform

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
@mberrys
mberrys merged commit 0704eef into dev Oct 5, 2026
16 checks passed
@mberrys
mberrys deleted the chore/issue-31-native-a11y-qualification branch October 5, 2026 22:34
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant