chore(ci): qualify native accessibility against the installed package (#31) - #200
Merged
Merged
Conversation
Run the product Quick accessibility qualification against a staged copy of the installed tree, not the developer build tree, on Linux and Windows. The native and software runs write separate records with distinct claims; Windows drives the native UI Automation client and requires it; Linux records the OS accessibility lane unavailable. scripts/ci/verify_quick_accessibility_evidence.py fails closed when a required lane is missing, failed, captured outside an installed tree, or replaced by a software-only record.
…lane on any platform Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Qualify native accessibility and the Widgets-free graph against the installed package (L03-05, #31). The MSI and AppImage workflows now run the product Quick accessibility harness against a staged copy of
build/install— the tree a user runs and the tree the Widgets-free profile is verified against — instead of the developer build tree, and the native/software distinction is structural in the record, not prose.Refs #31. Topic PR into
dev; the issue stays open until the promotion to the default branch (github closing keywords only act on a merge intostable).Where the qualification belongs, and why
The native claim cannot be made from a developer build tree. A build-tree run loads the build-tree
LoopLibCore.dll/LoopLibQuick.dlland resolves Qt fromLOOP_QT_ROOT; only the installed tree contains the deployed Qt closure that theverify-widgets-free-release-profile.py --install-dirscan actually blesses. The steps therefore stay exactly where they are in the job order — aftercmake --installand before packaging — but now stage the installed tree and run the harness from inside that copy.The harness binary is a qualification harness, not product surface: installing it would make
verify-loop-surface.ps1flag it as an unmanifested first-party artifact and would ship it in the MSI. Soscripts/lib/loop-quick-a11y-staging.ps1copiesbuild/installto a fresh staging directory and drops the harness in that copy'susr/bin. The shipped install tree is never mutated.What changed
scripts/run-product-quick-a11y-smoke.ps1—-InstallTreestages and runs against the installed closure;-Platformselects the Qt platform (Windows useswindows, i.e. real d3d11, not the offscreen software fallback); a non-offscreen native run fails if it falls back to the software rasterizer; each run writes a machine-readable record with the source SHA, installed-tree path, executable digest and observed backend.scripts/run-installed-quick-a11y-uia.ps1(new) — wires the previously-unwiredtools/ProductQuickAccessibilitySmoke/inspect-windows-uia.ps1driver (added by L02-04 — Expose accessible operator semantics #25) to the installed tree. This is the only lane that can prove the native accessibility backend is active:QAccessible::isActive()stays false until a real accessibility client attaches, which an offscreen harness run can never demonstrate.scripts/ci/verify_quick_accessibility_evidence.py(new) — fail-closed gate over the records. Chosen over adding a parallel evidence document: it tightens the verification the workflow already runs rather than introducing a second, reconcilable standard. A missing, skipped or failed native lane fails; a software-only record offered as the native claim fails because it carries a different claim and no OS-accessibility observation.LinuxInstall.yml/WindowsInstall.yml— native + software qualification against the installed tree, Windows additionally drives UI Automation and requires it, then the gate runs. Native stays fatal on both platforms.scripts/ci/test_verify_quick_accessibility_evidence.py(14 cases, incl. the software-in-native-slot case) and a newtest_workflow_contracts.pycase pinning the installed-tree wiring, the Windows-requires-native-accessibility asymmetry, and lane order.QUICK_ACCESSIBILITY_CONTRACT.md,ACCESSIBILITY_BASELINE.md,PACKAGING_LICENSING.md,CI.md.Evidence
Source SHA (base = first parent):
0f2f7599e3e17ce4e57bfdedfce85582c8f51115. Commit:f860d22f. Install-tree identity qualified locally:C:\.dev\repos\loop-build-31\install; harnessexecutable_sha256=2a740b974ec3702b81ee619582ef5229e54b46de0077217727149da4a9219a89, operator fixturefixture_sha256=31cc0710a652e2586131c1d87297f7a5988e68d79442b1e1d0f5b82ec2b9a49d.Exercised on this Windows host (
cmake --installintoC:/.dev/repos/loop-build-31/install, staged copy, Qt removed from PATH so only the installed closure resolves):-Platform windows)graphics_api=d3d11,artifact.scope=installed-treeQT_QUICK_BACKEND=software)graphics_api=software,stale_rejected=1on the finding-navigation fixturegraphics_api=d3d11,native_accessibility_backend_active=trueverify_quick_accessibility_evidence.pyover the three recordscheck-change.py --base origin/devstatus: pass, modules[documentation], all global guards green)check-architecture.py --base origin/devarchitecture contracts okA regression check confirms the base-tree lanes still behave as callers expect: build-tree native and software both PASS under
offscreen.Honest finding: the previous "native" lane was not native. Under
QT_QPA_PLATFORM=offscreenthe smoke reportedgraphics_api=softwareon both backends, so the old native evidence could not have supported a native claim. The installed-tree lanes now use-Platform windowson Windows and assert a non-software backend.Unavailable / not exercised:
quick-a11y-native-accessibility-unavailable.txtrather than leaving the lane absent-and-fine.inspect_package_dependencies.pyevidence.json. No package was installed locally — the staged install tree was qualified.Anti-slop review
The diff moves the accessibility qualification onto the installed tree and makes the native/software distinction structural, without restructuring any build contract. The accepted risk is that the Linux OS-accessibility lane is genuinely unavailable headless — recorded as unavailable rather than as a pass — while the Windows UI Automation lane was exercised locally (42 observations, d3d11) and is otherwise only proven by the hosted run. One duplicated harness locator remains between the smoke script and the staging lib; it is small and left for a later cleanup rather than widening this change.
Need help on this PR? Tag
@codesmith-botwith what you need. Autofix is disabled.