Skip to content

docs: explain grading-schema overrides - #1035

Closed
tomatotomata wants to merge 1 commit into
apache:mainfrom
tomatotomata:codex/document-grading-schema-951
Closed

docs: explain grading-schema overrides#1035
tomatotomata wants to merge 1 commit into
apache:mainfrom
tomatotomata:codex/document-grading-schema-951

Conversation

@tomatotomata

Copy link
Copy Markdown
Contributor

Closes #951

Summary

  • document the per-fixture grading-schema.json override in the release-prepare eval README
  • link the real step-1-plan/fixtures/grading-schema.json example
  • clarify that grading-schema.json controls exact-versus-rubric prose comparison while assertions.json supplies deterministic predicates

Validation

  • git diff --check
  • PYTHONPATH=tools/skill-evals/src python -m pytest tools/skill-evals/tests/test_runner.py -k grading_schema -q (5 targeted tests passed; the CLI-grader case is excluded)
  • the Windows CLI-grader test remains blocked by its existing shell-command fixture (WinError 2)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Document the grading-schema override in the release-prepare eval README

1 participant