Skip to content

Report tool-selection and argument-validity metrics #46

Description

@askmy-stack

Summary

Add reportable metrics for model-backed behavioral runs so teams can compare baseline and candidate tool interfaces.

Acceptance criteria

  • Report tool-selection accuracy by probe and aggregate
  • Report argument-validity rate and validation failures
  • Report risk and confirmation expectation compliance
  • Results are available in JSON and Markdown
  • Metrics distinguish missing data from failed evaluations
  • Tests use deterministic fixture results

Metadata

Metadata

Assignees

No one assigned

    Labels

    area:probesbehavioral contractsenhancementNew feature or requestpriority:p1Next product — M1 remainder + M3/M4 start

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions