Add a `SuppressedInferenceInvocation` class to handle nested `InferenceInvocation` operations by DylanRussell · Pull Request #765 · open-telemetry/opentelemetry-python-genai

DylanRussell · 2026-09-21T16:36:51Z

Description

Use class SuppressedInferenceInvocation in place of InferenceInvocation when an instrumentation starts an inference invocation that has already been started by a instrumentation higher up in the call stack..

Same logic that is in #663 but uses a new child class to represent inner invocations, makes it more clear what inner invocations are responsible for -- mostly just updating the context.

Fixes # (issue)

Type of change

New feature (non-breaking change which adds functionality)

How has this been tested?

Unit tests

Checklist

Followed the style guidelines of this project
Changelog updated if the change requires an entry
Unit tests added
Documentation updated

…ontext

…_context # Conflicts: # instrumentation/opentelemetry-instrumentation-genai-bedrock/tests/test_converse.py # instrumentation/opentelemetry-instrumentation-genai-bedrock/tests/test_invoke_model.py

…_context # Conflicts: # util/opentelemetry-util-genai/src/opentelemetry/util/genai/_inference_invocation.py # util/opentelemetry-util-genai/src/opentelemetry/util/genai/_invocation.py

…Invocation

opentelemetry-pr-dashboard · 2026-09-21T16:37:49Z

Pull request dashboard status

Waiting on the author · refreshed 2026-09-23 22:42 UTC

Resolve merge conflicts.

Respond to 3 review items (e.g. link a commit, explain why not, ask a follow-up):

Top-level threads: 1, 2, 3

Status above doesn't look right?

Just replied or pushed? Anything around or after the refresh time above may not be picked up yet — give it a few minutes.
Should this be with reviewers? Comment /dashboard route:reviewers to route it to them.
Anything wrong — including the routing? Report it with what you expected; it helps us improve the dashboard.

Copilot

Copilot review overview

🟡 Changes recommended

Critical issues remain in suppressed invocation construction and stream metadata processing.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 2 · 1 · 1

Open (4)

new redirection still runs the base initializer · New
Suppressing chunk processing loses response metadata · New
Use the captured context when publishing suppressed invocations · New
Duplicated initialization risks desynchronized invocation state · New

What changed in this PR

Adds suppressed nested inference invocations to prevent duplicate telemetry while preserving context and stream enrichment.

Changes:

Introduces SuppressedInferenceInvocation and context propagation.
Updates stream handling for nested invocations.
Adds tests, fixture isolation, Bedrock updates, and changelog coverage.

File	Summary
`util/opentelemetry-util-genai/tests/test_utils.py`	Updates span relationship tests.
`util/opentelemetry-util-genai/tests/test_stream.py`	Tests suppressed stream processing.
`util/opentelemetry-util-genai/tests/test_context.py`	Tests nested invocation propagation.
`util/opentelemetry-util-genai/src/opentelemetry/util/genai/stream.py`	Adjusts nested stream handling.
`util/opentelemetry-util-genai/src/opentelemetry/util/genai/invocation.py`	Exports the new invocation type.
`util/opentelemetry-util-genai/src/opentelemetry/util/genai/_invocation.py`	Adds context and lifecycle support.
`util/opentelemetry-util-genai/src/opentelemetry/util/genai/_inference_invocation.py`	Implements suppressed invocations and enrichment.
`util/opentelemetry-util-genai/src/opentelemetry/util/genai/_context.py`	Adds inference context helpers.
`util/opentelemetry-util-genai/.changelog/663.added`	Documents context propagation.
`util/opentelemetry-test-util-genai/src/opentelemetry/test_util_genai/fixtures.py`	Adds context isolation.
`instrumentation/opentelemetry-instrumentation-genai-bedrock/tests/test_invoke_model.py`	Updates Bedrock invocation tests.
`instrumentation/opentelemetry-instrumentation-genai-bedrock/tests/test_converse.py`	Updates Bedrock converse tests.

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

lmolkova

thanks for putting this together!

I'm thinking about an approach where we use strongly typed context data and consolidate invocations at finish time.

Pros:

Strict typing and explicit fields make precedence and validation much easier to reason about.
Clean and contained lifecycle: publish on inner finish, enrich on outer finish.
Plays nicely with Weaver codegen since the data container maps directly to schema models.

Cons:

More manual boilerplate to map fields today (Weaver codegen in PR #702 will generate most of this anyway).
Adding new semconv attributes requires updating the dataclass until codegen is in place.

Here is a proof of concept:
lmolkova@eb38238

lmk what you think

DylanRussell · 2026-09-22T14:17:22Z

That link doesn't work for me.

Precedence already seems very clear to me -- outer instrumentation always overrides inner. Inner only wins when outer hasn't set an attribute.

Typing what's in the context seems a little overkill to me, it will probably make it harder for instrumentations outside the repo to use it.

I dont think we gain much by typing it within the repo either -- it seems safe to assume the type and value is valid, our own code is typed already and just set the values. ultimately we just merge the two attribute maps and one map overrides the other.

lmolkova · 2026-09-22T20:01:56Z

sorry for the broken link, here's the right one lmolkova@7dcc59d

can you help me understand why and how instrumentation outside of this repo would care about internal details of suppression without taking dependency on util? It would need to put a very specific dict with very specific keys under internal context object. It's already tight enough coupling that would not let suppression work reliably if one of the libs does not use util.

What we gain is the same logic for the same properties. request.model becomes attribute on both - spans and metrics, no need for two different places to know that. We also don't need to copy things over between dicts of unknown nature.

DylanRussell · 2026-09-22T20:33:17Z

Both adk and litellm have native instrumentation that emit inference spans and will be installed alongside libraries we instrument in this repo that also emit inference spans.

Asking them to take a new dependency (which requires a newish version of the otel api/sdk) is a bigger ask than just reading / writing to the context which i think is straightforward without utils. Maybe they wouldn't do it in either case but we should verify that..

DylanRussell · 2026-09-22T20:42:56Z

I'll ask someone from adk to take a look

DylanRussell · 2026-09-22T20:47:35Z

Merging the attributes back into the class is nice to do

lmolkova · 2026-09-23T07:05:48Z

Both adk and litellm have native instrumentation that emit inference spans and will be installed alongside libraries we instrument in this repo that also emit inference spans.

Asking them to take a new dependency (which requires a newish version of the otel api/sdk) is a bigger ask than just reading / writing to the context which i think is straightforward without utils. Maybe they wouldn't do it in either case but we should verify that..

in order to participate in suppression, they would need to at least have access to the context key, which needs an implicit or explicit dependency. They can achieve suppression with minimal level of pain, but participating in merging attributes is an extra that I don't think is achievable without proper dependency on util. Having dictionaries with bespoke structure (even if documented) is implicit dependency with nothing guaranteeing any form of compatibility.

I'd prefer to draw a line here - suppression of inner layers can be done easily, merging is a cherry on top.

lmolkova · 2026-09-23T22:41:42Z

@DylanRussell created two side-by-side prototypes of native instrumentation that interacts with utils

using typed object
using dicts

97cadb5

Either way it's possible and relatively easy to do, but with optional / guarded importing it's more obvious what you do - we can keep the API public and enforce back-compat. With dictionaries everything is stringly type, brittle, and hard to enforce

DylanRussell and others added 26 commits September 9, 2026 20:42

Second version of the prototype

a44cffc

Merge remote-tracking branch 'origin/main' into prototype_adding_on_c…

e76c489

…ontext

style: format test_context.py

5f50205

Handle metric attributes coming from the context

d5eb19c

Remove context propagation helpers from public

bf9bd69

Merge remote-tracking branch 'origin/main' into prototype_adding_on_c…

84111ed

…ontext

Address comment

72f944d

Address error attributes comment

e58900b

Don't process chunk on inner invocations

d69c435

Fix bad tests add changelog entry

af06fe5

Merge remote-tracking branch 'upstream/main' into prototype_adding_on…

150370b

…_context # Conflicts: # instrumentation/opentelemetry-instrumentation-genai-bedrock/tests/test_converse.py # instrumentation/opentelemetry-instrumentation-genai-bedrock/tests/test_invoke_model.py

Simplify implementation

fe49a84

Simplify things even further

9b166c1

Simplify and generalize approach

c72afeb

Remove chunk parsing from inner

f738880

Inline some code

05aaab3

More simplification

f4feaba

Minor change

0cf715a

Merge branch 'main' into prototype_adding_on_context

205b336

Address comments

88c776f

Explain why we are doing things..

4cdd812

Merge branch 'main' into prototype_adding_on_context

44e676f

Clarify code

642ca8a

Merge remote-tracking branch 'upstream/main' into prototype_adding_on…

e916f2a

…_context # Conflicts: # util/opentelemetry-util-genai/src/opentelemetry/util/genai/_inference_invocation.py # util/opentelemetry-util-genai/src/opentelemetry/util/genai/_invocation.py

Remove already_started and should_capture_content from deprecated LLM…

03d17d2

…Invocation

Create SuppressedInferenceInvocation class

cc2f1c5

Copilot AI lite review requested due to automatic review settings September 21, 2026 16:36

DylanRussell requested a review from a team as a code owner September 21, 2026 16:36

github-project-automation Bot added this to GenAI Semantic Conventions and Instrumentation libraries Sep 21, 2026

Copilot started reviewing on behalf of DylanRussell September 21, 2026 16:37 View session

opentelemetry-pr-dashboard Bot mentioned this pull request Sep 21, 2026

Pull Request Dashboard #171

Open

Copilot AI reviewed Sep 21, 2026

View reviewed changes

DylanRussell mentioned this pull request Sep 21, 2026

Prototype of span / event / metric deduplication and context passing for inference span #663

Open

5 tasks

DylanRussell and others added 3 commits September 21, 2026 16:57

Address copilot

7bbac60

Remove unneeded overrides

cca0819

Merge branch 'main' into suppressed_inference_invocation

1fe618e

lmolkova reviewed Sep 22, 2026

View reviewed changes

Merge branch 'main' into suppressed_inference_invocation

1ad14b3

Merge branch 'main' into suppressed_inference_invocation

2719d76

This was referenced Sep 22, 2026

[opentelemetry-instrumentation-genai-agno] Add instrumentation for agno model calls #772

Closed

[opentelemetry-instrumentation-genai-agno] Add instrumentation for agno model calls #773

Open

DylanRussell added 2 commits September 22, 2026 20:07

Merge branch 'main' into suppressed_inference_invocation

a052d1b

Merge branch 'main' into suppressed_inference_invocation

6ebab61

Navigation Menu

Add a `SuppressedInferenceInvocation` class to handle nested `InferenceInvocation` operations - #765

Add a `SuppressedInferenceInvocation` class to handle nested `InferenceInvocation` operations#765
DylanRussell wants to merge 33 commits into
open-telemetry:mainfrom
DylanRussell:suppressed_inference_invocation

DylanRussell commented Sep 21, 2026 •

edited

Loading

Uh oh!

opentelemetry-pr-dashboard Bot commented Sep 21, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lmolkova left a comment

Uh oh!

DylanRussell commented Sep 22, 2026 •

edited

Loading

Uh oh!

lmolkova commented Sep 22, 2026

Uh oh!

DylanRussell commented Sep 22, 2026

Uh oh!

DylanRussell commented Sep 22, 2026

Uh oh!

DylanRussell commented Sep 22, 2026

Uh oh!

lmolkova commented Sep 23, 2026

Uh oh!

lmolkova commented Sep 23, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

DylanRussell commented Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

How has this been tested?

Checklist

Uh oh!

opentelemetry-pr-dashboard Bot commented Sep 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Pull request dashboard status

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Copilot review overview

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lmolkova left a comment

Choose a reason for hiding this comment

Uh oh!

DylanRussell commented Sep 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lmolkova commented Sep 22, 2026

Uh oh!

DylanRussell commented Sep 22, 2026

Uh oh!

DylanRussell commented Sep 22, 2026

Uh oh!

DylanRussell commented Sep 22, 2026

Uh oh!

lmolkova commented Sep 23, 2026

Uh oh!

lmolkova commented Sep 23, 2026

Uh oh!

This branch has not been deployed

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

DylanRussell commented Sep 21, 2026 •

edited

Loading

opentelemetry-pr-dashboard Bot commented Sep 21, 2026 •

edited

Loading

DylanRussell commented Sep 22, 2026 •

edited

Loading