Skip to content

fix(evals): recognize abbreviated assurance check claims - #171

Merged
vriesd merged 2 commits into
mainfrom
fix/delivery-assurance-claims
Oct 6, 2026
Merged

vriesd merged 2 commits into
mainfrom
fix/delivery-assurance-claims

Conversation

@vriesd

@vriesd vriesd commented Oct 6, 2026

Copy link
Copy Markdown
Contributor

Why

A live 9.6qualification rejected a truthful handoff that said "Assurance: completion supported, with all 4checks satisfied." The native close response recorded completion-supported and four satisfied checks. Adding only the adjective "assurance" made the unchanged full-input grader pass.

Scope

Make "assurance" optional before "checks" inside the two existing qualified-count branches of assuranceValue. Preserve the existing typed conclusion and count, capture groups, full-clause anchoring, and native count/status comparison. Add an exact retained-answer fixture and positive/adversarial regression coverage. Tests land before the fix.

Tradeoffs

Recognize the abbreviated noun only inside the existing assurance-value context. Standalone "all 4checks" stays outside this change. No general natural-language judge or new contextual state is introduced.

Blast Radius

This changes the delivery presentation grader. Runtime, guides, release thresholds and case catalog remain unchanged. The original stopped campaign stays officially 57/58 with 14 primary cells unexecuted. Original reports and outcomes are preserved; offline diagnostics do not qualify the release.

Verification

  • Actual retained full scenario input fails with one assurance issue before the fix and passes afterward.
  • New regression suite reproduced 19 pass/9 fail before the fix.
  • Parent and independent reviewer each passed 126 delivery tests with 387 assertions across four suites. Count mismatches, unsupported/negated conclusions, extra tails, contradictory second claims, historical text, Goals and registered commands remain guarded.
  • Separate development replay of all 58 unchanged native inputs passed 58/58. This is diagnostic evidence only.
  • Full clean-worktree contribution preflight passed 2,293 tests, with 20 optional live skips and zero failures. All 13 gated recorded cassettes reproduced.
  • Rebuilt package SHA-256 remains aa0ad68901d6c352a7956c327062328059ff6d7bbd234d933d0bec04dda781f3, exactly matching the stopped campaign artifact.
  • No additional paid calls or canary dispatch occurred.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-06T02:29:59.641324Z e33a717 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@vriesd

vriesd commented Oct 6, 2026

Copy link
Copy Markdown
Contributor Author

Independent shipping verification PASS at e33a717 against 7aa6c94. The reviewed source SHA-256 remains 36a8ac2d780308dc5186bfff9fea55de9710d29f2ed65411d446013476477c36; tests and retained fixture are unchanged from the independently verified test-first commit. The previous 126 passing tests and 387 assertions apply without drift. Checkout and diff are clean. Actual retained answer and original input hashes remain unchanged. Full consumption, native conclusion/count/status, authority, source and disclosure checks remain intact. This verifier-only result is not qualification. No paid calls, source edits or remote writes occurred.

@vriesd
vriesd merged commit 2af47b1 into main Oct 6, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants