Skip to content

[aw] Add trusted KBE search eval support - #133958

Draft
vitek-karas wants to merge 1 commit into
dotnet:mainfrom
vitek-karas:vitek-karas-kbe-search-prereq
Draft

vitek-karas wants to merge 1 commit into
dotnet:mainfrom
vitek-karas:vitek-karas-kbe-search-prereq

Conversation

@vitek-karas

@vitek-karas vitek-karas commented Sep 15, 2026

Copy link
Copy Markdown
Member

Summary

  • Add the repository-scoped KBE issue-search wrapper used by the CI scanner eval.
  • Add the candidate-read grader and focused Node tests.
  • Provision these evaluator inputs from the trusted base checkout and wire the focused test and grader into the eval runner.

Role in the stacked change

This is the prerequisite for dotnet/runtime#133684. It must merge before #133684 so the trusted evaluator files are available from main. The /ci-eval workflow restores evaluator inputs from main before checking out a pull request; without these files on main, the dependent PR's wrapper, grader, and focused tests cannot be trusted or exercised by the evaluator.

After this PR merges, #133684 can retain the production workflow, instruction, and eval-spec changes that consume this support.

Validation

  • node --test .github/workflows/evals/kbe-search.test.mjs — 10 tests passed.
  • git diff --check passed.

Note

This pull request description was generated by GitHub Copilot.

Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).
15 pipeline(s) were filtered out due to trigger conditions.
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot encountered an error and was unable to review this pull request. You can try again by re-requesting a review.

Note

This error may be related to your runner configuration. You can now configure runners for Copilot code review separately from Copilot cloud agent by creating a copilot-code-review.yml file with your setup steps. Read the docs for details.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area-skills Agent Skills

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants