Repository navigation
RL environments: Windows desktop packs (UI Automation and pixels-only RDP) - #1798
Merged
Merged
Conversation
… environments One generic browser environment (openadapt_flow.rl_env) serves every app over the OpenEnv protocol. The reward is verify_run's verdict on the app's record, read through a channel the agent's browser cannot reach; each episode also returns an outcome vector (completion, harm, false refusal, steps). - MockMed: 11-task public sample with hidden faults. MockLoan and the held-out MockMed faults ship no public tasks; they load from a pack kept outside the package (OPENADAPT_ENV_EXTRA_TASKS), and every pack is hashed. - OpenEMR (GPL-3.0), Frappe Lending (GPL-3.0) and openIMIS (AGPL-3.0): manifests name upstream images by digest; no app code is copied. OpenEMR has a task pack awaiting a live run; Frappe has its oracle; openIMIS has no reset yet. - A failure report on an impossible task pays only after the browser sent the task's write, so not trying never earns reward. - environments/prime/openadapt_web_envs: verifiers v1 taskset over the same servers.
- Public generator holds public cases only; held-out packs are built from a private cases file. The held-out instances in the previous commit are retired and re-authored privately. - Task ids are numbers and the observation carries no task id, so nothing the agent sees names the fault. - External apps (one shared database) serve one session at a time. - A failed reset leaves no live task or judge; a route-handler error can no longer hang a request; write detection compares trimmed values and numbers by value; the write_requests rule also covers extra packs; goto takes only paths; missing environment files raise TaskError. - run_eval wraps verifiers' eval: it never uploads traces without --allow-upload, and a test fails if our tooling calls eval directly.
…ble wrong_record cases MockMed's public sample passes over OpenEnv and the Prime path with zero model calls; 8 parallel sessions all reach the maximum reward; reset leaves no state behind. The generator now refuses a wrong_record task on the patient that fault writes to, which validation caught in a held-out pack.
…on Python 3.11+ The public artifact inventory now lists environments/openenv. On Python 3.10 the OpenEnv stack resolved to dnspython 2.8.0 (GHSA-45c3-73f7-pv4m); the extra now applies on 3.11+ only, which verifiers already requires.
…cord-checked apps Patient Chart (WinForms) and Claims Desk (WPF), our own apps on SQLite, served over the OpenEnv contract with verify-run's verdict as the reward. The guest is the user's Windows machine; reset restores the database from its pristine copy and refuses an episode whose digest differs from the seed. - rl_env/desktop.py: the environment (windows_uia, rdp_pixels surfaces). - rl_env/windows_guest.py: WinRM control plane (deploy, reset, dump). - rl_env/validate_desktop.py: scripted, screen-trusting, planted-fault and random agents, state-leak check, reset timing. - win_agent /uia/tree and WindowsBackend.window_tree: read-only window element list for the UIA observation. - SessionProfile.gfx: ask FreeRDP 3 for H.264/AVC444. - 24 public tasks per app; a held-out half of the same shape is private.
From an independent review of the first commit (one blocker): - Keys are an allowlist after normalization (no Windows key, Alt or function keys; Ctrl only with editing shortcuts). The denylist let cmd+r and control+escape through. - The apps run full screen over the taskbar. - The pack's code is read-only to the desktop session (data in C:\oa\pack\data); deploy proves it with a write probe from the desktop and refuses the guest otherwise; every reset checks pack_db.py's hash. - Provisioning sets NoWinKeys, NoRun and DisableTaskMgr. - A failed reset leaves no episode to act on or judge. - Reset stops every desktop-session process except the shell and the agent, and any save still writing. - Values reaching PowerShell are plain identifiers; quoting covers the Unicode single quotes; window_tree checks its strings and depth.
- Reset stops only the desktop account's own processes in its own session (owner and session from query session), never system processes. - Oracle-side PowerShell runs with -NoProfile and the system modules only; Python runs with -I, so nothing the desktop account writes is loaded. - Optional separate desktop account (OPENADAPT_WINENV_DESKTOP_USER). - The write probe has a positive control and fails closed if it never ran; deploy removes an explicit grant left by an earlier setup. - Identifiers use fullmatch; a failed seed is a GuestError. - Claim tasks say the amount is typed exactly as written.
…fact inventory) - pack_db logs each save the app asks for, before any fault, cleared at every seed; an infeasible task pays a failure report only after a real try, as on the web environments. - Register the packs' yaml/json files in public-artifacts.json. - The web contract test skips app.kind desktop directories (they load through rl_env.desktop and have their own tests).
…s-on-1780 # Conflicts: # packages/flow/openadapt_flow/backends/xfreerdp.py # packages/flow/tests/test_xfreerdp.py
…family - WindowsGuest reaches a guest over SSH with a key when one is configured (else WinRM); settings fall back to the lab profile (OAFLOW_<PROFILE>_*). - OPENADAPT_WINENV_RESET_SCOPE=pack stops only the pack's processes, for a desktop shared with other work. - Task ids are opaque and tags never name the fault (the agent sees the id). - Three feasible tasks per half lose their first save while the screen says saved; the reference solution re-opens the record and saves again. - The RDP surface uses main's SessionProfile(avc444=...) (#1789).
… to FreeRDP with AVC444
OPENADAPT_WINENV_{SSH,AGENT,RDP}_VIA=host:port lets the server run where it
cannot reach the guest's network directly (an RDP client container behind a
relay). The pixels-only environments ask FreeRDP 3 for H.264/AVC444;
OPENADAPT_WINENV_RDP_TRANSPORT=aardwolf runs them from any host.
- Element offsets (at) are fractions of the element's box: on a 200% display the fixed pixel offset hit the grid's header, not its first row. - Over SSH, probe cleanup of a missing file no longer reads as a failure.
Found on the zenbook AVC444 lane: openbox, started after the fullscreen client window, gave it a title bar. The remote desktop was then drawn about 20 px below the frame's coordinates and every click landed that much high (a grid's first row hit its header). -decorations keeps the window bare. Also: the desktop environment passes the session profile by keyword.
A script read with ReadToEnd from Windows OpenSSH's stdin sometimes waited forever (seen twice on zenbook). Short scripts now go as one encoded command line with no stdin; longer ones (cmd.exe caps a command line at 8191 characters) go after their length, and the reader stops at the last character. Each call times out after 180 s instead of 600.
WPF's DataGrid raises no SelectionChanged for the row already selected, so re-opening a claim to check a save left the old form filled in; the amount typed again was appended and refused. Found by the three lost-first-save tasks on zenbook (21 of 24 reference solutions passed before the fix).
As agreed with #1780: AppSpec.kind gains desktop, with surface and a desktop block (guest app, window, readiness, RDP session, guest image) validated only for that kind. The desktop environment loads through load_environment and serves the guest's records with the shared _serve_records, read by verify.yaml as ${RECORDS_URL}/records. server.build_app dispatches desktop manifests; the browser environment refuses them. Desktop manifests list their checks. The write_requests rule does not apply to desktop apps, whose save log tells a try from none.
All four environments pass every check on a real guest: 24/24 reference solutions pay 1, 12/12 feasible tasks are refuted with a planted fault, random 0.0, screen-trusting 0.5 (the feasible half), no state leaks. Reset median 6.6-7.3 s (UI Automation) and 3.2-3.3 s (RDP, AVC444 negotiated). The first Claims Desk run (21/24, the re-open bug) is kept.
…round From the third independent review: - Blocker: the held-out half could be rebuilt from make_tasks.py (public seed, deterministic build); its hash matched the private file. It now comes from its own random stream with a secret seed kept beside it in the private repository, and its note tags no longer carry the plan index. The public half is byte-identical, so the validation evidence still matches. - One source per guest's settings: OPENADAPT_WINENV_* when its HOST is set, else the lab profile; a lab key or relay never reaches another host. - The account reset scope fails when the desktop account has no session or its processes' owners cannot be read; a domain account matches in full. - Validation output records the reset scope.
From the fourth review round (nits): an account spelled differently from its process owners (.\name, name@domain) matched nothing and the account reset passed while stopping nothing; it now refuses. An explicit host argument takes its other settings from OPENADAPT_WINENV_* too.
70 runs per build on the Windows 11 x64 laptop, aardwolf from the Mac, rdp_avc_real's protocol: 40 saves, 20 false stops (decoy 10, 1440x900 10), 10 stops on the look-alike, 0 wrong records, on both builds, with identical stop messages. The false stops come from a postcondition on an auto-selected row and a refused session size, not from identity checks.
…s-on-1780 # Conflicts: # packages/flow/environments/openenv/README.md # packages/flow/openadapt_flow/rl_env/environment.py # packages/flow/openadapt_flow/rl_env/server.py # packages/flow/openadapt_flow/rl_env/tasks.py # packages/flow/public-artifacts.json # packages/flow/pyproject.toml # packages/flow/tests/e2e/test_rl_env_e2e.py # packages/flow/tests/test_rl_env.py # packages/flow/uv.lock
abrichr
marked this pull request as ready for review
November 4, 2026 07:43
Results under results/windows11-x64-2026-11-04/ with a short summary of the four environment validations and the RDP false-stop comparison. The A/B metadata names the real client and guest. The task manifests say only that the held-out half is private. The superseded pre-fix Claims Desk run is dropped.
The guest's host, user and WinRM password are non-empty strings or a clear GuestError, never None or blank into a PowerShell command. Also: the validator read the surface from the old manifest field (it would have raised at the end of a run), the RDP transport is typed for both clients, and close() calls each teardown explicitly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #1780. Merge that first; this PR's base is
feat/openenv-benches.What this adds
Four OpenEnv environments on two Windows line-of-business apps, with the same task,
verify.yamland reward contract as #1780. The reward is verify-run's verdict on the app's SQLite database, read outside the agent's session.win-patient-chartwin-patient-chart-rdpwin-claims-deskwin-claims-desk-rdpprovision/setup_guest.ps1prepares that machine.drop,wrong_row,duplicate,truncate,stale). Every target has a look-alike record. The UIA and pixels-only variants share one task set. A held-out half of the same shape is kept privately;tasks/MANIFEST.jsonrecords its SHA-256.rl_env/validate_desktop.pyruns four agents (scripted, screen-trusting, planted-fault, random), a state-leak check and reset timing.Runtime and backend changes (reviewed)
win_agent: a new read-only/uia/treeoperation (bounded; returns no field values).WindowsBackend.window_treeis its client.xfreerdp.SessionProfile.gfx: requests H.264/AVC444 from FreeRDP 3. Existing profiles keep their digest.An independent review ran in two rounds. It found one blocker: the agent could reach a shell (a key-denylist bypass, plus the Start menu) and rewrite the code the oracle runs. It's fixed by:
pack_db.pyon every reset;-NoProfileandpython -Ion the oracle side;Also fixed: a failed reset no longer leaves the previous episode open, the reset cleanup is limited to the desktop account's own processes, and inputs are validated before they reach PowerShell.
The desktop environments also follow #1780's attempted-write rule: the app logs each save request before any fault, and an honest failure report pays only after a real attempt.
Not in this PR yet
environments/openenv/_windows/results/before this leaves draft.Tests
tests/test_rl_env_desktop.pycovers:pack_db.py, the REST shim and verify-run;Also updated:
test_win_agent_server.py(/uia/tree),test_xfreerdp.py(gfx), andtest_rl_env.py, which now skipsapp.kind: desktopdirectories.