Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions packages/flow/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@

### Fixes

- Self-repair on a covered identity band dismisses only the proved dialog: the covering window must span the band from where the read stops to its right edge at full height, the dismiss click must land in that same window, and only a dialog whose every word is a notice word (no question) may be dismissed ([#1724](https://github.com/OpenAdaptAI/OpenAdapt/issues/1724)).

- Self-repair can take a stop where the app's own dialog covers the next step's identity band: when the backend proves the covering window belongs to the window the last verified step was in, and the part of the band read still matches the record, the dialog is dismissed after the previous click and the step re-checks its identity in full. Another record behind the dialog stays an identity halt. The runtime lets a backend remember a verified record's window (`record_verified_window`); macOS answers `owned_window_over` from the window's proved sheet ([#1724](https://github.com/OpenAdaptAI/OpenAdapt/issues/1724)).

- macOS: a run can now act in the pinned window's own AppKit sheet (an `NSAlert` attached to it), which is a separate window. `MacOSBackend` accepts it only when AX lists exactly one `AXSheet` on the window and exactly one window of the same process has its bounds inside it; the capture composites exactly that sheet over the window, and focus, pointer and click-target checks accept it only inside its bounds. Any other window is still refused ([#1711](https://github.com/OpenAdaptAI/OpenAdapt/issues/1711)).
Expand Down
25 changes: 24 additions & 1 deletion packages/flow/docs/REPAIR_ON_EVERY_BACKEND.md
Original file line number Diff line number Diff line change
Expand Up @@ -156,15 +156,38 @@ when all of these hold:
(`record_verified_window` at that step, then `owned_window_over`). On
macOS that is the window's one proved sheet; on Windows an owned window of
the same process, first over the band in the live z-order;
- that window is what cut the band: its rectangle spans the band from where
the text read stops to the band's right edge, at the band's full height
(the share of the band left uncovered is within 0.1 of the share of the
recorded text read). A popup that only touches the band, such as a tooltip
over a column's ellipsis, does not count;
- the part of the band that was read still matches the recorded record: no
contradicted, look-alike, glyph-ambiguous or unexplained name token, and at
least half the band matched. Another patient's header behind the dialog
fails this and stays an identity halt;
- the previous step is a click that doesn't save, acted on a trusted rung.

The dismiss click is held to that dialog. It must land in the very window
`owned_window_over` reported (asked again at the click point), and the
dialog's text must read as a notice only. Every word in it must be on a
short list of session-notice words ("Your session expires in 5 minutes",
"OK"), and it shows no punctuation beyond `. , : !`, so it asks no question
in any script. It must also say that a session expires, so a dialog whose
message wasn't read (only its "OK" was) is refused, and so is any text that
runs across the dialog's edge. The click's label must be "OK". Anything else refuses,
including an update prompt, a dialog in another language, or one with
nothing readable in it.
Answering "OK" to "Discard unsaved changes?" would lose data. The bridge is
exactly that one click. A covered-band repair gets one attempt: a later
attempt reopens the app, whose dialog is a new window, so its click is
refused.

Nothing after that is relaxed: the probe replays the halted step with its own
identity check in full once the dialog is gone, then the record check and a
person decide as for any repair.
person decide as for any repair. That full check is what protects the case
where the dialog hides exactly where two records differ (a same-name patient
with another birth date): the read beside the dialog can match, and the
recheck refuses.

## One lifecycle on every backend

Expand Down
7 changes: 6 additions & 1 deletion packages/flow/openadapt_flow/backends/macos_backend.py
Original file line number Diff line number Diff line change
Expand Up @@ -948,7 +948,12 @@ def owned_window_over(
sx, sy, sw, sh = rect
x, y, w, h = (int(v) for v in region)
if sx < x + w and x < sx + sw and sy < y + h and y < sy + sh:
return {"kind": "sheet", "window_id": sheet[0], "pid": window.pid}
return {
"kind": "sheet",
"window_id": sheet[0],
"pid": window.pid,
"region": [int(sx), int(sy), int(sw), int(sh)],
}
return None

def _attached_sheet(
Expand Down
Loading
Loading