feat(go): Go SDK with OpenAI instrumentor - #166
Conversation
Scaffold the Go SDK with OTel TracerProvider setup, GenAI semantic convention keys, and the Instrumentor interface. Signed-off-by: pmady <pavan4devops@gmail.com>
HTTP middleware that intercepts chat/completion/embedding calls and records them as OTel spans with GenAI semconv attributes. Signed-off-by: pmady <pavan4devops@gmail.com>
Covers chat completion happy path, non-AI endpoint passthrough, error response handling, and content capture toggle. Signed-off-by: pmady <pavan4devops@gmail.com>
Signed-off-by: pmady <pavan4devops@gmail.com>
Signed-off-by: pmady <pavan4devops@gmail.com>
|
this covers finding 7 from #164. middleware approach means no hard dep on the openai-go library -- any HTTP client hitting their API paths gets traced. the module structure mirrors what python does (core + per-framework packages) so adding anthropic/cohere later is straightforward. cc @niaborowy @JayaSurya-27 |
|
@JayaSurya-27 any thoughts on the go sdk? lmk if the approach works or if you'd rather do it differently |
|
Hi team, just checking in .Happy to address any feedback on this PR. |
abhijaisrivastava15
left a comment
There was a problem hiding this comment.
Hi @pmady,
Thank you for contributing to Future AGI.
I read through the whole thing. The layout mirrors the Python side (core package plus a per-framework package), the HTTP exporter points at the right ingest path, you pinned semconv/v1.26.0 so it matches the SDK's default resource schema and Register does not error on a schema conflict, and the request-parameter and token attributes use the canonical gen_ai.* keys. That is a solid base to build on.
This is a whole new language SDK though, so before we can move it forward two things need to happen: a maintainer needs to own the decision to ship a Go SDK now (the roadmap note in #164 is a pointer, not a sign-off), and the span schema needs to match what our platform actually ingests, because right now several spans will be exported but will not map. Concretely:
-
Content keys. You emit
gen_ai.promptandgen_ai.completion. Our canonical schema keys prompt and response content offgen_ai.input.messages.<i>.role/.contentandgen_ai.output.messages, plusinput.value/output.valuefor the raw blob (see the Pythontraceai_openaiextractor). As written, the prompt and completion text will not land in the platform's input/output, and evals that read those will not bind. -
Span kind. The spans never set
gen_ai.span.kind. The Python instrumentors set it toLLM, and the platform uses it to classify a span as an LLM call. Without it these render as generic spans. Please set it, and confirm in the UI that a real trace shows up as an LLM call. -
The content-capture toggle drops too much. With
WithContentCapture(false)the middleware skips both the request and response bodies entirely, so you also lose the model name, token usage, response id and finish reasons, not just the prompt and completion text. Those are not PII. Please gate onlygen_ai.prompt/gen_ai.completionbehind the toggle and always capture model and usage. -
Streaming. For a streamed call the response body is an SSE stream.
io.ReadAllbuffers the whole stream before returning it, which defeats streaming for the caller, and the SSE text is not a single JSON object soextractResponseAttributesfails and tokens, response model and completion are silently dropped. Please detect the streaming case and handle it, or at minimum skip the read so you do not break the caller's stream. -
gRPC transport is misconfigured.
WithEndpointexpectshost:portwith no scheme (example.com:4317), but you passhttps://grpc.futureagi.com. UseWithEndpointURLfor a full URL, orgrpc.futureagi.com:443withWithEndpoint. As is, the gRPC path will not connect, andconfig.gohas no test so nothing catches it. -
The module is not consumable as published.
traceai_openai/go.modrequirestraceai v0.0.0with a localreplace ../traceai.replaceis ignored for anyone importing your module, andv0.0.0is not a real tag, sogo getof the openai package will fail outside this repo. This needs a real tagged version of the core module and arequirethat points at it.
On tests: the core traceai module has none, so go test ./... passes there only because there is nothing to run. Please add a test around exporter and endpoint construction to pin the transport and header wiring, and add streaming and content-capture-off cases to the middleware tests so the model and token attributes are asserted.
Smaller things, non-blocking: gen_ai.operation.name uses completion and embedding where the semconv values are text_completion and embeddings; gen_ai.request.duration_ms duplicates the span duration; and the Instrumentor interface and Tracer helper in instrumentor.go are unused, with Tracer calling the deprecated trace.NewNoopTracerProvider.
Happy to do a full pass once the span schema matches the platform and the module is go-gettable.
Thank you
Addresses review feedback on future-agi#166. - set gen_ai.span.kind=LLM so the platform classifies these as LLM calls - WithContentCapture(false) now gates only gen_ai.prompt/completion; model, token usage, response id and finish reasons are always recorded - gRPC exporter uses WithEndpointURL, since FI_GRPC_URL carries a scheme and WithEndpoint expects host:port - gen_ai.operation.name uses the semconv values text_completion and embeddings - drop gen_ai.request.duration_ms, which duplicated the span duration - delete the unused Instrumentor interface and Tracer helper; Tracer returned a noop provider despite its comment claiming it fell back to the global one - add exporter tests covering the ingest path, auth headers and the gRPC URL - assert model and usage survive content capture being turned off - gofmt config.go and semconv.go Still open, pending your call: the input/output content keys (1), streaming (4) and tagging the core module (6).
Sets gen_ai.span.kind so spans classify as LLM calls, scopes WithContentCapture to prompt/completion only so model and usage survive it, and switches the grpc exporter to WithEndpointURL. Adds exporter tests for the ingest path and auth headers. Drops the unused Instrumentor interface and Tracer helper. Signed-off-by: pmady <pavan4devops@gmail.com>
|
@abhijaisrivastava15
three i'd rather not guess at:
and the bigger one: who owns the ship/don't-ship call on a go sdk? happy to keep going, just don't want to keep pushing if there's no appetite for it in-tree yet. |
What does this PR do?
Adds a Go SDK for traceAI. Two modules:
go/traceai— core setup (TracerProvider config, GenAI semconv keys, env-based defaults)go/traceai_openai— HTTP middleware for theopenai-goclient that captures chat/completion/embedding calls as OTel spansWhy?
Go SDK was listed on the roadmap (#164 finding 7). Lots of Go services calling LLM APIs directly or through gateways, and there was no Go instrumentation path.
How was it tested?
go test ./...passes in both modules (4 tests covering happy path, passthrough, error, content capture toggle)go vet ./...cleanChecklist
mainNotes for reviewers
5 commits over ~9 days. The middleware operates on
net/httptypes so it doesn't pull in the openai-go library as a dependency — works with any HTTP client that hits the OpenAI API path patterns.Related: #164, #165