[opentelemetry-instrumentation-genai-agno] Add instrumentation for agno model calls - #772
Closed
DylanRussell wants to merge 14 commits into
Conversation
Instrument agno.models.base.Model response and streaming methods to emit standard GenAI chat spans with complete request/response attributes, usage metrics, and content capture. Assisted-by: Claude Opus 4.6
Replace Any with concrete Agno Model, Message, MessageData, and ModelResponse types in model patching, stream wrapping, and provider resolution functions. Assisted-by: Claude Opus 4.6
…tent variable Remove redundant capture_content local variables and function parameters, relying directly on invocation.should_capture_content property on InferenceInvocation and RetrievalInvocation. Assisted-by: Claude Opus 4.6
Replace repetitive try/except type conversions with safe_int and safe_float helpers across retrieval, model inference, and streaming wrappers. Assisted-by: Claude Opus 4.6
…hecks Let opentelemetry-util-genai handle None values directly across model response token metrics, embedder tokens, dimension count, and query text. Assisted-by: Claude Opus 4.6
Avoid repeated checks of getattr(model, 'vertexai', False) and GOOGLE_GENAI_USE_VERTEXAI by caching the boolean in is_vertex. Assisted-by: Claude Opus 4.6
Eliminate repetitive is_vertex if/else branching by determining google_provider once at function start. Assisted-by: Claude Opus 4.6
Consolidate duplicate provider resolution logic across resolve_embedder_provider and resolve_model_provider into a single _resolve_provider helper. Assisted-by: Claude Opus 4.6
…_model_output_message Drop unnecessary cast(object, ...) and cast(list[Any], ...) calls and share tool call part extraction via _extract_tool_call_parts. Assisted-by: Claude Opus 4.6
…nput_messages Use list[dict[str, Any]] | None for tool_calls and Iterable[Message] for messages, accessing Message attributes directly. Assisted-by: Claude Opus 4.6
…n embedder functions Use concrete Model | Embedder and Embedder annotations in _resolve_provider, resolve_embedder_provider, and embedder wrappers. Assisted-by: Claude Opus 4.6
…ment) Assisted-by: Claude Opus 4.6
… | ModelResponse Assisted-by: Claude Opus 4.6
Pull request dashboard statusClosed · refreshed 2026-09-22 14:45 UTC Status above doesn't look right?
|
DylanRussell
added a commit
that referenced
this pull request
Sep 22, 2026
Assisted-by: Claude Opus 4.6
Assisted-by: Claude Opus 4.6
DylanRussell
force-pushed
the
DylanRussell/more_agno_instrumentation_5
branch
from
September 22, 2026 14:38
c21ed35 to
a7100e1
Compare
DylanRussell
changed the base branch from
main
to
DylanRussell/more_agno_instrumentation_4
September 22, 2026 14:38
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
Unresolved critical and moderate findings affect invocation safety, telemetry duplication, structured results, schema conformance, and failure coverage.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 2
Open (2)
What changed in this PR
Adds Agno GenAI instrumentation for model, embedding, retrieval, workflow, and tool operations, with shared invocation and streaming utility updates.
Changes:
- Adds Agno instrumentation, lifecycle handling, and provider conversions.
- Adds retrieval-document models and serialization support.
- Expands tests, conformance coverage, documentation, dependencies, and changelogs.
| File | Reviewed change |
|---|---|
util/opentelemetry-util-genai/tests/test_toolcall.py |
Tests invocation context lifecycle. |
util/opentelemetry-util-genai/tests/test_stream.py |
Tests tool-stream lifecycle. |
util/opentelemetry-util-genai/tests/test_handler_retrieval.py |
Tests retrieval-document serialization. |
util/opentelemetry-util-genai/src/opentelemetry/util/genai/utils.py |
Extends JSON serialization utilities. |
util/opentelemetry-util-genai/src/opentelemetry/util/genai/types.py |
Adds RetrievalDocument; its public metadata type uses Any instead of the required JSON type (nit). |
util/opentelemetry-util-genai/src/opentelemetry/util/genai/stream.py |
Adds stream wrappers; structured chunks are stringified and concatenated instead of preserved (moderate). |
util/opentelemetry-util-genai/src/opentelemetry/util/genai/_retrieval_invocation.py |
Adds retrieval serialization; public document typing uses Any (nit), and deep-copy serialization can fail during finalization (moderate). |
util/opentelemetry-util-genai/src/opentelemetry/util/genai/_invocation.py |
Adds suspension and activation; unguarded detach can replace caller errors and leave spans unended (critical). |
util/opentelemetry-util-genai/.changelog/673.added |
Documents shared utility additions. |
instrumentation/opentelemetry-instrumentation-genai-llama-index/tests/requirements.oldest.txt |
Adds workflow test dependency. |
instrumentation/opentelemetry-instrumentation-genai-llama-index/tests/requirements.latest.txt |
Adds workflow test dependency. |
instrumentation/opentelemetry-instrumentation-genai-agno/tests/test_workflow_background.py |
Tests background workflows; early-close coverage lacks final status and error assertions (moderate). |
instrumentation/opentelemetry-instrumentation-genai-agno/tests/test_user_id.py |
Tests user and session IDs. |
instrumentation/opentelemetry-instrumentation-genai-agno/tests/test_tools.py |
Tests tool definitions and streaming; adds a prohibited type: ignore (nit). |
instrumentation/opentelemetry-instrumentation-genai-agno/tests/test_stream.py |
Tests agent, team, and workflow streams. |
instrumentation/opentelemetry-instrumentation-genai-agno/tests/test_model.py |
Tests model instrumentation; sync and async stream failure paths remain untested (moderate). |
instrumentation/opentelemetry-instrumentation-genai-agno/tests/test_knowledge.py |
Tests retrieval instrumentation. |
instrumentation/opentelemetry-instrumentation-genai-agno/tests/test_instrumentor.py |
Tests instrumentation lifecycle. |
instrumentation/opentelemetry-instrumentation-genai-agno/tests/test_embedder.py |
Tests embedding instrumentation; several wrapper failure paths remain untested (moderate). |
instrumentation/opentelemetry-instrumentation-genai-agno/tests/test_conformance.py |
Registers embedding and retrieval scenarios; chat and tool execution operations lack conformance scenarios (moderate). |
instrumentation/opentelemetry-instrumentation-genai-agno/tests/test_agent.py |
Tests continuation and agent attributes. |
instrumentation/opentelemetry-instrumentation-genai-agno/tests/requirements.oldest.txt |
Adds OpenAI test dependency. |
instrumentation/opentelemetry-instrumentation-genai-agno/tests/requirements.latest.txt |
Adds OpenAI test dependency. |
instrumentation/opentelemetry-instrumentation-genai-agno/tests/conformance/retrieval.py |
Adds retrieval conformance scenario. |
instrumentation/opentelemetry-instrumentation-genai-agno/tests/conformance/embedding.py |
Adds embedding conformance scenario. |
instrumentation/opentelemetry-instrumentation-genai-agno/src/opentelemetry/instrumentation/genai/agno/utils.py |
Adds Agno conversions and provider helpers; emits tool_calls instead of normalized tool_call (moderate). |
instrumentation/opentelemetry-instrumentation-genai-agno/src/opentelemetry/instrumentation/genai/agno/stream.py |
Adds Agno model and tool stream wrappers; can create duplicate inference spans and mishandle structured chunks (moderate). |
instrumentation/opentelemetry-instrumentation-genai-agno/README.rst |
Documents supported operations; model chat documentation can advertise duplicate telemetry (moderate). |
instrumentation/opentelemetry-instrumentation-genai-agno/.changelog/738.added |
Documents embedder instrumentation. |
instrumentation/opentelemetry-instrumentation-genai-agno/.changelog/737.added |
Documents IDs and background workflows. |
instrumentation/opentelemetry-instrumentation-genai-agno/.changelog/731.added |
Documents continuation methods. |
instrumentation/opentelemetry-instrumentation-genai-agno/.changelog/673.added |
Documents retrieval and tool streaming. |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| def __init__( | ||
| self, | ||
| stream: Any, | ||
| invocation: InferenceInvocation, |
Comment on lines
+395
to
+409
| def test_model_aresponse_stream( | ||
| instrument_agno, | ||
| span_exporter, | ||
| ) -> None: | ||
| """Test that Model.aresponse_stream emits an async streamed chat span.""" | ||
| model = _StreamModel(id="async-stream-model", provider="Anthropic") | ||
| messages = [Message(role="user", content="Async stream prompt")] | ||
|
|
||
| async def _run() -> list[Any]: | ||
| results = [] | ||
| async for chunk in model.aresponse_stream(messages=messages): | ||
| results.append(chunk) | ||
| return results | ||
|
|
||
| chunks = asyncio.run(_run()) |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Probably want to wait until #765 lands to avoid duplicate inference calls, but eventually we will want this and that PR will not change the implementation here.
I think once this lands we will have reached parity with the
agnoinstrumentations in open inference and other places..Type of change
How has this been tested?
Unot tests
Checklist