Skip to content

fix(evaluation): remove public llm-grader assertion#1730

Merged
christso merged 3 commits into
mainfrom
av-ij5o-remove-llm-grader
Jul 7, 2026
Merged

fix(evaluation): remove public llm-grader assertion#1730
christso merged 3 commits into
mainfrom
av-ij5o-remove-llm-grader

Conversation

@christso

@christso christso commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • reject public authored eval assertions using type: llm-grader with migration guidance to llm-rubric or agent-rubric
  • keep the internal shared LLM grader runtime plumbing used behind public rubric assertion types
  • update SDK public assertion helpers plus docs/skills/examples to teach llm-rubric instead of llm-grader

Validation

  • bun run build
  • focused core tests for schema/validator/parser/transpiler/yaml/jsonl/source-traceability: 401 pass, 0 fail
  • bun test packages/sdk/test: 100 pass, 0 fail
  • bun run validate:eval-schema
  • git diff --check

Live dogfood was not run because this only changes public authored schema/parser/docs rejection and SDK authoring helpers; grader runtime behavior and live LLM grading execution are unchanged.

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 7, 2026

Copy link
Copy Markdown

Deploying agentv with  Cloudflare Pages  Cloudflare Pages

Latest commit: 154dbf2
Status: ✅  Deploy successful!
Preview URL: https://dd18053b.agentv.pages.dev
Branch Preview URL: https://av-ij5o-remove-llm-grader.agentv.pages.dev

View logs

@christso christso merged commit 27bd6ed into main Jul 7, 2026
9 checks passed
@christso christso deleted the av-ij5o-remove-llm-grader branch July 7, 2026 14:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant