[abrichr]

Part of OpenAdaptAI/openadapt-internal#102 (the code change; a pre-registration is tracked there).

What changed

• The reward contract can now score the case where the required effect is present but the policy asked a person instead of declaring done. Until now that case was unscored. New outcome escalated_after_effect, new scoring class partial_credit.

• New field escalated_after_effect_reward, default 0.5, range [0, 1), finite. A validator refuses it unless it is below verified_reward. It is part of the contract digest. The docstring and REWARD.md explain that the value sets the policy's threshold for declaring done rather than asking: (x - w) / (v - w), which is 75% with the defaults (1, -1, 0.5).

• judge_episode: after a halt, refuse or policy-reject signal, if every required effect is CONFIRMED and no forbidden effect is present, the outcome is escalated_after_effect with no uncertainty.

  • A forbidden effect still gives wrong_effect.
  • Any other required effect that is present (wrong or duplicate) still gives reconciliation_required, unscored. That keeps the existing handling for an escalation after a wrong write.

• Evidence receipts: a partial-credit receipt must have a scalar of 0 or more and carry no uncertainty, the same rule as verified.

• JSON schemas (reward-contract-v1.json, reward-evidence-receipt-v1.json), types/docs/REWARD.md and flow/docs/REWARD_WORKER.md are updated to match.

• Harness² read-back scorer (integrations/harness2/scorer.py): it fell through to the -0.5 wrong-effect penalty for the new outcome, which CI caught in test_harness2_adapter.py.

  • The search reward for escalated_after_effect is now the signed receipt's own scalar, the contract's value. With no scalar on the receipt it stays unscored.
  • The mockmed/timeout task now also accepts escalated_after_effect. A stop after the one right save used to come out as reconciliation_required, which the task accepted. A wrong, stray or duplicate write still never passes.
  • docs/HARNESS2.md and the worker's OutcomeLiteral now list the outcome.

How it was tested

• types: 770 passed, 2 skipped. New tests:

  • the default scalar for every scored outcome
  • accepted values 0, 0.25, 0.5 and 0.999
  • out-of-range values (negative, 1, above 1, NaN, inf) are refused
  • the value must stay below verified_reward
  • the value changes the contract digest
  • a receipt is never negative or uncertain

• flow reward tests (tests/test_reward*.py): 71 passed. New tests: each halt signal with the effect confirmed gives partial credit; a wrong write after a halt is still reconciliation; a forbidden effect after a halt is still wrong_effect.

• flow ruff check, ruff format --check, mypy (589 files) and check_consistency.py pass.

• Harness²: tests/test_harness2*.py and tests/test_reward*.py give 104 passed, 6 skipped. The 6 skips need the optional harness2 package, which isn't installed locally; CI runs them. Changes and new tests:

  • The halt-after-the-right-save row now expects escalated_after_effect with 0.5 (it was reconciliation_required, unscored).
  • New: a stop after the one right save passes the timeout task, with reward 0.5.
  • New: an escalation whose receipt has no scalar stays unscored.
  • With the old scorer, all three fail at -0.5.
  • Ruff, format and mypy pass on the changed modules.

What couldn't be tested

• The full flow suite didn't finish locally.
  • The first run (pytest -n 6 tests) hung for more than 60 minutes.
  • A rerun with --timeout 600 reached about 12%. The only errors by then were in tests/test_desktop_check_it_cli_mockmed.py: its setup times out.
  • That file shows the same setup timeouts on main (87547bd), so this PR didn't cause them.
  • The tests that use the reward code all pass on this branch: test_reward_worker, test_reward_trust_boundary, test_narration_rules, test_undo_proposal and test_probe_induction (150 passed, 1 skipped). The types tests also pass (770 passed, 2 skipped).
  • CI runs the full suite now that this is marked ready.
• No training run uses the new value yet. That is the separate pre-registered sweep.

[abrichr]

Independent adversarial review: APPROVE

From: Codex. Reviewed 1cdf013 in a fresh detached worktree with fresh locked environments.

The finite, below-verified policy checks (packages/types/openadapt_types/reward.py:303–318) and the all-required-confirmed gate after forbidden/indeterminate checks (packages/flow/openadapt_flow/reward/worker.py:174–218) hold. No unsafe outcome gained positive credit in the probes.

Validation: Types installs without Flow/cv2/onnxruntime; 770 tests pass, 2 skip. Affected Flow reward/Harness and adjacent suites: 184 pass, 7 skip. Types Ruff/oldest-syntax check and Flow Ruff/format/mypy pass (591 sources). Twelve independent probes pass: real SQLite partial receipts for every stop signal, client scalar handling, signature tampering, custom-credit certificate binding, invalid/uncertain receipts, missing baselines and harmful-write precedence.

The optional Harness² package was not installed, so its package integration cases skip. Tests used synthetic public fixtures, headless browsers and the Linux resource gate. No training result is claimed. I did not change or push the reviewed branch.