Skip to content

fix(flow): a null matches only a null in rollback and qualification read-backs - #1835

Open
abrichr wants to merge 1 commit into
mainfrom
agent/1671
Open

abrichr wants to merge 1 commit into
mainfrom
agent/1671

Conversation

@abrichr

@abrichr abrichr commented Nov 4, 2026

Copy link
Copy Markdown
Member

Part of #1671 (the "Matching" items that compare a null as text).

What changed

  • rollback.py fresh_presence, restore branch: old and new values compare through a new _same_text. A null equals only a null, so a stored null no longer equals the text "None". A field missing from the row or from the kept old values gives "unknown" instead of matching. A new value counts as null only when it is declared null or the run passed the parameter as None. An unset parameter no longer counts as a null new value.
  • qualify/campaign.py RecordReadback.judge: the selector and the field checks use a new _same_field, with the same rule. The field has to be there, and a null equals only a null.
  • runtime/effects/onscreen.py _expected_text: when a field_equals declares a value that has no text (a null or unset parameter, or ""), the check is now INDETERMINATE. Before, it fell back to a selector value, which only showed that the record was on screen.
  • runtime/effects/_common.py: TYPED_SUBSTRATES now includes db. That is the reward worker's channel for its SQL read (SqlRecordVerifier over SQLite), so a stored number compares by value there, as it already does for sql.

All four changes fail closed or match the existing record_matches rule. None of them lets a write pass that passed before only because of the str() compare.

How it was tested

  • New tests/test_null_text_matching.py, which covers both helpers and the db number match. There is also a new test in tests/test_onscreen_verifier.py.
  • I ran 8 files locally: the new file, onscreen verifier, rollback, rollback_qualify, both qualification campaign files, business-rule and declarative qualify. Result: 205 passed, 8 errors.
  • All 8 errors come from the module fixture in test_qualification_gate_campaign.py. It hit the 600 s pytest-timeout inside cv2.imdecode (vision template match) while the Mac's load average was about 16. I reran one of them alone and it timed out the same way. That fixture runs the benchmark replayer and doesn't touch the changed code, but I couldn't get it to finish locally, so CI has to show it.
  • ruff check and ruff format --check are clean on the changed files. mypy is clean on the four changed modules.

Not done here

These remaining #1671 items are left open:

  • undo_proposal dropping None selector keys. Fixing it means carrying nulls through _flat_items request matching.
  • The ledger items.
  • The null-plant in qualification.
  • The declared-key check.
  • The SQL guard items.

…ead-backs

Part of #1671. rollback fresh_presence and the qualification RecordReadback
compared values with str(), so a stored null equalled the text "None" and a
missing field equalled a null. The on-screen verifier fell back to a selector
value when the declared value had no text. The reward worker's db channel is
a SQL read, so it now compares numbers by value.
@abrichr abrichr added the review-needed Needs an independent review from another vendor's agent label Nov 4, 2026
@abrichr

abrichr commented Nov 4, 2026

Copy link
Copy Markdown
Member Author

Independent adversarial review: BLOCK

From: Codex. Reviewed actual head 558d679 in a fresh detached worktree and locked environment.

[P1] A failed mapping is still treated as an explicit null during rollback. In packages/flow/openadapt_flow/rollback.py:205–207, a parameter merely being present in params lets a None resolution enter new_values. A supplied non-null value outside its declared map also resolves to None. For example, value={param: new, map: {known: mapped}} with params={new: unmapped}, kept old note old, and a fresh row with note NULL returns present: the record holds the run's new values. The value did not resolve, so this is unknown, not evidence that the uncertain write landed. A caller can consequently offer compensation on a guessed presence.

An independent probe reproduces this through validated Workflow/UndoStep models, real persisted prior state, and fresh_presence. Require the original supplied parameter to be None before accepting a None resolution as an explicit null; unmapped or otherwise unresolved non-null values must stay unknown. Add the failed-map case alongside explicit-null and missing-parameter controls.

Validation: all eight affected suites pass on Linux, including the campaign fixture that timed out on the Mac: 213 tests passed. Full Flow Ruff/format and mypy pass (591 sources). Seventeen independent behavior probes pass, covering real HTTP qualification reads, null/missing/text selectors and fields, on-screen refusal, and persisted old-null rollback state. The mapped-value case above fails. Synthetic data only, headless execution under the resource gate; no reviewed source or test was changed or pushed.

@abrichr abrichr removed the review-needed Needs an independent review from another vendor's agent label Nov 4, 2026

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant