fix: give native graph spans their LaunchDarkly identity by apucacao · Pull Request #121 · launchdarkly/python-ai-sdk

apucacao · 2026-09-30T22:49:03Z

Summary

Spans from the native graph adapters (to_openai_agents, to_claude_agents, to_lang_graph) had nothing tying them to the graph's AI Config, so Monitoring couldn't find them by config key.

The launchdarkly.graph span now gets set_ld_span_attributes, like every other handler: config key, run id, context keys and the feature_flag event. New helper: make_graph_track_data.
Graph and node tracking events now carry the environment id when LD_ENVIRONMENT_ID is set.
Graph-level events ($ld:ai:graph:invocation_success and friends) now use the graph key as configKey, matching the span and graph(). They used the root node's key.
The graph() runner's own launchdarkly.graph span gets the same identity.
make_graph_track_data stays internal (imported from launchdarkly_ai_server.utils, not in __all__). TELEMETRY-CONTRACT.md section 10 says which config key graph spans and events carry.
A setup error (an agent that fails to build, a graph that fails to compile) now fails the run like a run error does: every adapter opens the span before setup, records the exception with ERROR status, fires $ld:ai:graph:invocation_failure and ends the span once. Before, OpenAI and LangChain exported no span, and no adapter tracked the failure.

JS twin: launchdarkly/js-ai-sdk#101.

Testing

Adapter and client tests cover the span attributes, the event and the environment id.
Staging: graph run with this branch.

🤖 Generated with Claude Code

Note

Overview
Graph telemetry is aligned so Monitoring can tie traces to the graph AI Config.

Every launchdarkly.graph span—from graph() and the OpenAI, Claude, and LangChain native adapters—now goes through set_ld_span_attributes with a new make_graph_track_data payload (config/graph key, run id, feature_flag event, context keys). Graph-level track events (invocation_success, duration:total, total_tokens, invocation_failure) use the graph key as configKey instead of the root node’s key. Node events still use per-node keys.

The SDK graph() runner gets the same span identity, issues a fresh runId per invoke/stream, and passes run track data into cached routes so handoff events match that run. Tracking payloads include environmentId when LD_ENVIRONMENT_ID is set.

Native adapters wrap setup and execution in one try/finally: setup failures (agent build, graph compile, etc.) record an ERROR span, emit $ld:ai:graph:invocation_failure keyed to the graph, and end the span once. TELEMETRY-CONTRACT.md documents the contract.

^{Reviewed by Cursor Bugbot for commit a5e9ad3. Bugbot is set up for automated code reviews on this repo. Configure here.}

Every $ld:ai:* event that execute_and_track sends carries the LaunchDarkly environment id, because AI Config Monitoring needs it to match a trace to the config that produced it. The two payloads built outside that path did not: make_track_data, used by every native graph adapter, and the graph-level graph_track_data in graph.py. Both now read the id through the same _try_get_environment_id helper, and omit the key when there is no id to report. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

A native graph adapter has no track data of its own: make_track_data describes one node, and the run as a whole is the graph flag. Add make_graph_track_data, which names the graph flag as both the config key and the graph key, the same choice the SDK's own graph runner makes. Adapters pass it to set_ld_span_attributes in the next commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

to_openai_agents, to_claude_agents and to_lang_graph each open a launchdarkly.graph span that carried only the graph key and token counts. The AI Config Monitoring tab finds a trace by a feature_flag span event and the config key, so none of these runs appeared against their AI Config. Tag the span through set_ld_span_attributes, the same helper every handler already uses, so the graph span now carries launchdarkly.config.key, launchdarkly.variation.key, launchdarkly.run.id, the context keys, and the feature_flag event with feature_flag.set.id. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

apucacao · 2026-10-01T14:10:20Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

jeffdupont

Reviewed with the 1.0 freeze in mind. The fix is right: native graph spans now carry the config identity Monitoring needs, and the environmentId handling matches execute_and_track. Tests pass locally on 6c19b28 (1407 passed, 11 skipped).

Two things I'd like settled before GA, because they are hard to change once 1.0 ships:

New public name. make_graph_track_data is exported from the package root. The 1.0 API-surface plan makes make_track_data internal and renames it make_node_track_data, so this one should be internal too. Same for makeGraphTrackData in launchdarkly/js-ai-sdk#101. Details inline.
Graph span and graph events disagree on the config. The span now says config.key = <graph key>. The $ld:ai:graph:invocation_success / duration:total / total_tokens events in all three adapters still use make_track_data(root, ...), so their configKey is the root node's key. The SDK's own graph() uses the graph key for those events (graph.py _build_graph). That's not new in this PR, but event payloads are a data contract with Monitoring, and changing them after GA would quietly shift customers' dashboards. Could we pick one, either here or in a follow-up before GA? The monorepo TESTING.md doesn't say which configKey graph events carry, so it should be written down there too.

Smaller notes:

The SDK's own graph() span (graph.py:919 invoke, :1135 stream) still doesn't get set_ld_span_attributes, and neither does JS graph.ts:722. After this PR, native-adapter graph spans can be found by config key but graph() spans can't. Is that intended, or does Monitoring reach graph() traces through the node spans?
launchdarkly.graph.key is now set twice on the native span: once directly, then again by set_ld_span_attributes. It's harmless; the direct call could go.
Not from this PR: in openai-agents the span starts before the try that ends it, so a ValueError during agent setup (the "not built" checks) leaves the span un-ended and never exported. JS #101 adds an "ends the span once" test. Python may want the same.
I didn't find a matching ai-sdks-monorepo spec PR. The span attributes and feature_flag event on launchdarkly.graph are worth adding to TESTING.md so both SDKs stay in step.

jeffdupont · 2026-10-02T15:56:34Z

+    track_data: dict[str, Any] = {
+        "runId": run_id,
+        "configKey": graph_key,
+        "variationKey": "",


variationKey is always "" here (and version always 1), while graph() reports the real values from the graph flag's meta. Native-graph spans will always have an empty launchdarkly.variation.key, so per-variation Monitoring views won't work for them. If GraphDefinition exposes meta later, this signature (graph_key, run_id) has to change. That's one more reason to keep the helper private until then.

The launchdarkly.graph span that graph() opens, on invoke and on stream, carried only launchdarkly.graph.key. It now goes through set_ld_span_attributes with the same track data its graph events use, so it has the config key, variation, run id, context keys and the feature_flag event, like the native adapters' graph span. TELEMETRY-CONTRACT.md section 10 now says which config key the graph span and the graph-level events carry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

…ternal The $ld:ai:graph:invocation_success, invocation_failure, duration:total and total_tokens events from the three native adapters carried the root node's config key, while their graph span and graph() use the graph key. They now use make_graph_track_data, so a graph run reports the same config key whichever runner produced it. make_graph_track_data is no longer exported from the package root. Only the adapters use it, and its signature will change if GraphDefinition gains the graph's variation metadata, so it stays out of the 1.0 surface. The adapters no longer set launchdarkly.graph.key by hand, since set_ld_span_attributes already does. to_openai_agents and to_lang_graph now open the graph span only after the agents and graph are built. A setup error such as a child agent that was not built used to leave a started span that was never ended or exported. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

apucacao · 2026-10-02T17:13:32Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

graph() caches the built graph per context so repeated runs skip re-evaluating every node's flag. The build also created the graph's track data, run id included, so every run for the same context reported the same runId on its $ld:ai:graph:* events. Each run now copies the cached track data with a fresh runId and passes it to route and stream_route through their opts, so the node, handoff and graph-level events of one run share one id and the next run gets another. The flag evaluations stay cached.

A setup error in a native graph adapter (an agent that fails to build, a sub-agent tool that fails to build, a graph that fails to compile) was not reported as a graph failure. to_openai_agents and to_lang_graph opened the launchdarkly.graph span only after setup, so no span was exported at all, and none of the three adapters tracked $ld:ai:graph:invocation_failure for it. to_claude_agents also recorded a run error and ended its span twice, once in the run's own handler and again in the outer one. All three adapters now open the span, tag it with set_ld_span_attributes, and run setup and the run in one try. Any error records the exception on the span, sets status ERROR with the error message, tracks invocation_failure with the graph's track data when a context is present, and re-raises. The span ends exactly once, in a finally, so a failed setup still leaves no span open.

apucacao · 2026-10-05T22:45:55Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

apucacao · 2026-10-09T15:39:42Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

jeffdupont

Re-reviewed at 2bd05aed. Everything from my 2 Oct review is fixed except one thing, and it's the one that matters for GA.

Fixed, and checked at this head:

make_graph_track_data is gone from __all__; the adapters import it from launchdarkly_ai_server.utils.
Graph-level events (duration:total, total_tokens, invocation_success / invocation_failure) are keyed to the graph in all three adapters. Removing the failure track fails 3 tests.
graph() spans now carry their identity (graph.py:942, :1174), with a test for both modes.
launchdarkly.graph.key is no longer set twice.
A setup error now ends the adapter span with ERROR and sends invocation_failure (8c67043). Removing either part fails a test.

Suite: uv run pytest 1550 passed, 11 skipped; mypy and ruff clean. It merges cleanly with main.

Every graph() run with the same context gets the same run id (inline at :942). The id is created in _build_graph (graph.py:127-128), and that build is cached per context (:926-935 invoke, :1159-1167 stream). So a second conversation with the same user reports the first one's run id, and the two look like one run in LaunchDarkly. I ran it: invoke+invoke, stream+stream and invoke+stream each give one id on both spans and both invocation_success events. JS fixed the same bug in c83137e7 (graph.ts:752, passed to streamRoute at :795). The node closures also spread the cached graph_track_data into the handoff events (:271, :289, :447, :467, :563, :592, :706, :737). So the fix has to create the id per call and pass that copy down to the route, not just set it on the span. Could you add the two-runs test that monorepo #47 :757 asks for, covering spans and events?

Still open, my call, not blocking: the variationKey '' / version 1 fallback (utils.py:650-651). It's documented in TELEMETRY-CONTRACT.md and listed as a known gap in #47.

jeffdupont · 2026-10-12T22:57:26Z

        tracer = trace.get_tracer("@launchdarkly/ai-server")
        with tracer.start_as_current_span("launchdarkly.graph") as span:
-            span.set_attribute("launchdarkly.graph.key", self._key)
+            set_ld_span_attributes(


graph_track_data here comes from the per-context cache (:935), so its runId is the one _build_graph made the first time (:127-128). A second invoke with the same context sets the same launchdarkly.run.id and sends the same runId in invocation_success. I reproduced it in all three combinations: invoke+invoke, stream+stream and invoke+stream. Suggest doing what JS c83137e7 does: run_track_data = {**graph_track_data, "runId": str(uuid.uuid4())} per call, used here, in the events, and passed into the route so the handoff events get it too. Same at :1174 for stream.

## Summary Spans from the native graph adapters (`toOpenAIAgents`, `toClaudeAgents`, `toLangGraph`, `toVercelAgents`) had nothing tying them to the graph's AI Config, so Monitoring couldn't find them by config key. - The `launchdarkly.graph` span now gets `setLdSpanAttributes`, like every other handler: config key, run id, context keys and the `feature_flag` event. New helper: `makeGraphTrackData`. - Graph and node tracking events now carry the environment id. - Graph-level events (`$ld:ai:graph:invocation_success` and friends) now use the graph key as `configKey`, matching the span and `graph()`. They used the root node's key. - The `graph()` runner's own `launchdarkly.graph` span gets the same identity. - Every adapter now ends its graph span in a `finally`. `toLangGraph` never ended it, and `toOpenAIAgents` / `toClaudeAgents` left it open on setup errors. - `makeGraphTrackData` is marked `@internal`. - A setup error (an agent that was not built, a graph that fails to compile) now fails the run like a run error does: the span records the exception with ERROR status, and `$ld:ai:graph:invocation_failure` fires. Before, the span ended with no status and no failure event. - Each `graph()` run gets its own run id. The built graph is cached per context, so runs with the same context used to share one id on the span and every graph event. This replaces #109. - `toVercelAgents` no longer sends `$ld:ai:graph:path`, matching the other adapters. The path stays on the span. Python twin: launchdarkly/python-ai-sdk#121. ## Testing - Adapter and client tests cover the span attributes, the event and the environment id. The LangGraph span-end tests fail on main. - Staging: [graph run with this branch](https://ld-stg.launchdarkly.com/projects/default/traces?query=launchdarkly.operation.type%3Dgen_ai+launchdarkly.graph.key%3Dtravel-agent-flow&page=1&relativeTime=last_24_hours&env=production&env=staging&env=alexis-o11y-lab&selected-env=alexis-o11y-lab&related_resource=eyJ0eXBlIjoidHJhY2UiLCJpZCI6IjczYzhjYzU0NWIxNDYxNjg0YWVkMzRjNGU3MDg0MWVkIiwidGltZXN0YW1wIjoiMjAyNi0wOS0zMFQyMzoyNDo1OC45N1oiLCJzcGFuSUQiOiJjNTVmZWFiNjljZjliZDgwIn0%3D). 🤖 Generated with [Claude Code](https://claude.com/claude-code)  --- > [!NOTE] > **Overview** > Native graph runs and the client's `graph()` runner now tag **`launchdarkly.graph`** spans with the same LaunchDarkly identity as other handlers via **`setLdSpanAttributes`** and a new **`makeGraphTrackData`** helper, so AI Config Monitoring can correlate traces by graph/config key (including **`feature_flag`** span events with environment id). > > **Telemetry behavior changes:** graph-level `$ld:ai:graph:*` events and **`invocation_failure`** use the **graph flag key** as `configKey` (not the root node); **`environmentId`** is added to graph and node track payloads via **`tryGetEnvironmentId`**. **`graph().stream`** mints a **fresh `runId` per invocation** even when the built graph is cached. Setup failures (compile/build) are handled like run errors: span ERROR status, exception recorded, graph-level failure event, and **`span.end()`** once in **`finally`** (fixes missing/double span lifecycle in LangGraph and other adapters). **`toVercelAgents`** stops emitting **`$ld:ai:graph:path`** as a track event (path remains on the span only). > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit e41fe83. Bugbot is set up for automated code reviews on this repo. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup>  --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

Calls the same runner twice with the same context in each native adapter and checks that each run's launchdarkly.graph span carries the runId of that run's graph-level events, and that the two runs carry different ids.

apucacao · 2026-10-13T17:12:30Z

bugbot run

cursor

Stale Bugbot comment from a previous run.

Each run's launchdarkly.graph span must carry the runId of that run's graph-level events, for both invoke and stream, and the two runs' spans must carry different ids.

apucacao · 2026-10-14T14:38:22Z

bugbot run

cursor

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 27f1447. Configure here.}

jeffdupont · 2026-10-14T17:56:47Z

The run-id blocker here (and Bugbot's graph.py:944 finding) is fixed by #132. I verified it: review. Merge #132 into this branch and I'll approve this one.

jeffdupont

#132 is in (head is now a5e9ad3, where I verified TestGraphRunId passes for invoke and stream). The run-id blocker is fixed, and Bugbot's graph.py:944 finding with it. CI green. The variationKey '' / version 1 fallback stays as the documented known gap in ai-sdks-monorepo#47.

apucacao and others added 3 commits September 30, 2026 18:46

apucacao mentioned this pull request Sep 30, 2026

fix: give native graph spans their LaunchDarkly identity launchdarkly/js-ai-sdk#101

Merged

cursor Bot reviewed Oct 1, 2026

View reviewed changes

apucacao marked this pull request as ready for review October 1, 2026 14:27

apucacao requested review from andrewklatzke and jeffdupont October 2, 2026 13:48

jeffdupont reviewed Oct 2, 2026

View reviewed changes

apucacao and others added 2 commits October 2, 2026 12:26

cursor Bot reviewed Oct 2, 2026

View reviewed changes

This was referenced Oct 2, 2026

fix: pass AI Config model parameters through to every provider handler #107

Open

feat(AIC-3408): add Vercel AI SDK messages and agents adapters #103

Merged

feat: add LiteLLM messages and agents integrations #102

Merged

apucacao added 2 commits October 2, 2026 16:31

cursor Bot reviewed Oct 5, 2026

View reviewed changes

jeffdupont mentioned this pull request Oct 8, 2026

fix(client): Agent Skills pre-release must-fix findings #148

Merged

Merge main into fix/native-graph-span-identity

2bd05ae

cursor Bot reviewed Oct 9, 2026

View reviewed changes

Merge fix/native-graph-span-identity into fix/graph-run-id-per-run

30f7d4a

apucacao added this pull request to stack #151 October 9, 2026 15:58

apucacao mentioned this pull request Oct 9, 2026

fix(client): give each graph() run its own run id #132

Merged

jeffdupont reviewed Oct 12, 2026

View reviewed changes

test(native-graph): each runner call gets its own run id

297e19c

Calls the same runner twice with the same context in each native adapter and checks that each run's launchdarkly.graph span carries the runId of that run's graph-level events, and that the two runs carry different ids.

Merge fix/native-graph-span-identity into fix/graph-run-id-per-run-clean

4ff4133

cursor Bot reviewed Oct 13, 2026

View reviewed changes

apucacao added 3 commits October 13, 2026 10:14

test(graph): check the span's run id in the two-runs test

4965404

Each run's launchdarkly.graph span must carry the runId of that run's graph-level events, for both invoke and stream, and the two runs' spans must carry different ids.

Merge main into fix/native-graph-span-identity

27f1447

Merge fix/native-graph-span-identity into fix/graph-run-id-per-run-clean

a5e9ad3

cursor Bot reviewed Oct 14, 2026

View reviewed changes

Comment thread packages/client/src/launchdarkly_ai_server/graph.py

apucacao requested a review from jeffdupont October 15, 2026 18:13

jeffdupont approved these changes Oct 15, 2026

View reviewed changes

apucacao merged commit e863b6d into main Oct 15, 2026
12 checks passed

apucacao deleted the fix/native-graph-span-identity branch October 15, 2026 18:14

github-actions Bot mentioned this pull request Oct 15, 2026

chore: release main #166

Open

Conversation

apucacao commented Sep 30, 2026 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Testing

Uh oh!

apucacao commented Oct 1, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeffdupont left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jeffdupont Oct 2, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

apucacao commented Oct 2, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

apucacao commented Oct 5, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

apucacao commented Oct 9, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jeffdupont left a comment

Choose a reason for hiding this comment

Uh oh!

jeffdupont Oct 12, 2026

Choose a reason for hiding this comment

Uh oh!

apucacao commented Oct 13, 2026

Uh oh!

cursor Bot left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

apucacao commented Oct 14, 2026

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

jeffdupont commented Oct 14, 2026

Uh oh!

jeffdupont left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

apucacao commented Sep 30, 2026 •

edited by cursor Bot

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading

cursor Bot left a comment •

edited

Loading