Skip to content

fix(evals): bind definite script integrity clauses - #180

Merged
vriesd merged 2 commits into
mainfrom
fix/command-integrity-article
Oct 7, 2026
Merged

vriesd merged 2 commits into
mainfrom
fix/command-integrity-article

Conversation

@vriesd

@vriesd vriesd commented Oct 7, 2026

Copy link
Copy Markdown
Contributor

Why

Qualification rejected a truthful handoff that said node scripts/verify.mjs passed with exit code 0; the script is unchanged. The command parser accepted only the possessive its script wording.

What changed

  • Recognize the bounded definite script clause as a script integrity claim.
  • Retain period-separated script and invocation qualifiers in the registered command record so invalid qualifiers cannot escape validation.
  • Preserve the actual failed answer in a focused fixture and cover both accepted wording and rejected claims.

Scope

Evaluator parsing and regression tests only. Native immutable script, source, validation, and review bindings still determine whether a claim is supported. Product package, release policy, case catalog, and passing requirements stay unchanged.

Tradeoffs

The parser accepts the demonstrated script wording without inferring invocation identity from it. It retains the existing closed grammar for other claims.

Blast Radius

This affects delivery grading. It changes no published plugin behavior. The stopped campaign remains failed and supplies no qualification credit.

Verification

  • The unchanged retained answer failed before the fix, passed when only the became its, and passes unchanged after the fix.

  • RED regression commit precedes GREEN. Focused checks pass with negative controls for changed scripts, missing evidence, incorrect exits, and unsupported qualifiers.

  • All ten retained reports remain byte-identical. Diagnostic replay accepts 342 of 344 attempts, preserves one genuine disclosure failure, and excludes one host failure.

  • Full push preflight passed with 2,858 tests passing, 20 existing skips, and zero failures. Independent source and package review passed on the exact head.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 7, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-07T22:28:27.694518Z b420e0e PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@vriesd
vriesd merged commit 336cddc into main Oct 7, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants