Skip to content

fix(evaluator): preserve boolean flags through cascade evaluation - #572

Merged
codelion merged 1 commit into
algorithmicsuperintelligence:mainfrom
Yushu-Li:fix/cascade-boolean-metrics
Nov 1, 2026
Merged

codelion merged 1 commit into
algorithmicsuperintelligence:mainfrom
Yushu-Li:fix/cascade-boolean-metrics

Conversation

@Yushu-Li

@Yushu-Li Yushu-Li commented Nov 1, 2026

Copy link
Copy Markdown
Contributor

Problem and change

Successful cascade stages can return numeric scores alongside boolean flags. The threshold fallback counts those flags as numbers, and merging stages converts them to floats. For example, {"score": 0.1, "valid": True} passes a 0.5 threshold and can acquire inflated fallback fitness after a merge. A false flag can also block an otherwise passing score.

I exclude boolean flags from fallback cascade thresholds and preserve their type when merging stages 1/2/3, so the existing fitness helper continues to ignore them. Numeric scores still normalize to floats. Numeric combined_score precedence, inclusive thresholds, error filtering, and timeout policy are unchanged. This is a separate successful-cascade path from the timeout work in #446.

Validation

  • Seven new test methods exercise actual local cascade modules, stage advancement, final fallback fitness, boolean-only results, and numeric controls. The regression cases fail on the original implementation.
  • New tests plus existing cascade validation: 18 passed.
  • Full documented command OPENAI_API_KEY=test-key-for-unit-tests python -m unittest discover tests: 997 tests, OK.
  • Black checks and git diff --check passed.

No paid model calls were made. This does not add general validation of invalid combined_score values or change the policy for failed later stages.

AI assistance: I used OpenAI Codex for implementation, tests, and automated review.

@codelion
codelion merged commit 6080971 into algorithmicsuperintelligence:main Nov 1, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants