Repository navigation
Add agentest - #155
Add agentest#155srpadmanabhuni wants to merge 1 commit into
Conversation
Reviewer's GuideDocumentationAdds agentest to the LLM & MCP Testing section as an offline, deterministic agent-evaluation tool.
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
Code Review by Qodo🐞 Bugs (0) 📘 Rule violations (0) 📎 Requirement gaps (0)
Great, no issues found!Qodo reviewed your code and found no material issues that require reviewTip of the day💡 Did you know, you can add REVIEW.md to your repo root and Qodo follows it on every PR |
PR Summary by QodoList agentest under LLM & MCP Testing
AI Description
Merge Risk
|
There was a problem hiding this comment.
Hey - I've found 1 issue
Prompt for AI Agents
Please address the comments from this code review:
## Individual Comments
### Comment 1
<location path="README.md" line_range="85" />
<code_context>
## LLM & MCP Testing
+- [agentest](https://github.com/srpadmanabhuni/agentest) - Offline, deterministic agentic evals for AI agents: 165 tests of OpenTelemetry GenAI traces, A2A AgentCards and MCP tool lists, with no LLM judge.
- [agent-convergence-scorer](https://github.com/hermes-labs-ai/agent-convergence-scorer) - A CLI and Python library that scores how lexically similar N agent or LLM outputs are: exact-match rate, Jaccard token overlap, divergence point, and a composite convergence score over any list of runs. An eval primitive for measuring reproducibility and fan-out collapse. Zero deps.
- [mcp-server-fuzzer](https://github.com/Agent-Hellboy/mcp-server-fuzzer) - A comprehensive fuzzing tool designed specifically for testing Model Context Protocol (MCP) servers. It supports both tool argument fuzzing and protocol type fuzzing across multiple transport protocols.
</code_context>
<issue_to_address>
**Testing tools appear out of order**
When the section is expected to remain alphabetized, the `agentest` entry appears before `agent-convergence-scorer`, leaving the section out of alphabetical order for readers.
Move the `agentest` entry after `agent-convergence-scorer`.
</issue_to_address>Sourcery assessment
Approved.
|
|
||
| ## LLM & MCP Testing | ||
|
|
||
| - [agentest](https://github.com/srpadmanabhuni/agentest) - Offline, deterministic agentic evals for AI agents: 165 tests of OpenTelemetry GenAI traces, A2A AgentCards and MCP tool lists, with no LLM judge. |
There was a problem hiding this comment.
⚪ Low · Testing tools appear out of order
When the section is expected to remain alphabetized, the agentest entry appears before agent-convergence-scorer, leaving the section out of alphabetical order for readers.
Move the agentest entry after agent-convergence-scorer.
Prompt for AI agents
In `README.md` at line 85:
**Testing tools appear out of order**
When the section is expected to remain alphabetized, the `agentest` entry appears before `agent-convergence-scorer`, leaving the section out of alphabetical order for readers.
Move the `agentest` entry after `agent-convergence-scorer`.There was a problem hiding this comment.
LlamaPReview — Blocking issues found
The new listing breaks alphabetical order in the section.
Owner action: Move the agentest listing after agent-convergence-scorer.
Review details and evidence
| Priority | File | Finding | Evidence |
|---|---|---|---|
| P2 | README.md |
The new listing breaks alphabetical order in the section. | confirmed |
LlamaPReview checks
- Reviewed changed regions in
README.md. - Read the complete PR-head file
README.md.
LlamaPReview is open source. Explore the code behind this review.
|
|
||
| ## LLM & MCP Testing | ||
|
|
||
| - [agentest](https://github.com/srpadmanabhuni/agentest) - Offline, deterministic agentic evals for AI agents: 165 tests of OpenTelemetry GenAI traces, A2A AgentCards and MCP tool lists, with no LLM judge. |
There was a problem hiding this comment.
P2 | Confidence: High
agent-convergence-scorer alphabetizes before agentest, so the addition breaks the requested ordering. The listing otherwise has the requested section and period-ending format.
Evidence: changed region in README.md; PR-head read of README.md.
Adds agentest to the LLM & MCP Testing section, in alphabetical order, following the contributing format (one entry, description ends with a period).
I am the author of agentest. This adds a single listing for the open-source runner.
The PyPI package name is agentest-ai; the import and CLI are agentest. The project is Apache-2.0, offline, and deterministic (165 tests, no LLM judge).
Happy to adjust the wording or section if needed.
Summary by Sourcery
Add agentest to the LLM and MCP testing resources list.
New Features:
Documentation: