JAVA-6222 Support trace context propagation to the server by nhachicha · Pull Request #2022 · mongodb/mongo-java-driver

nhachicha · 2026-07-23T11:53:10Z

JAVA-6222
This is the reference implementation for the Spec DRIVERS-3454 Support trace context propagation to the server specifications#1966
This PR also bump the wire protocol to 29 per https://jira.mongodb.org/browse/JAVA-6164

Spec proposal (OP_MSG section kind 3, W3C traceparent, hello tracingSupport negotiation) plus driver-sync POC design to validate it.

…r-sync)

…tion, wire-version gate)

…mpled with flags 00

…version gate supersedes)

Executed end-to-end against mongodb-mongo-master ee3a67f8 (9.0.0-alpha0): ServerSpanLinkageProseTest passes. Confirmed real server parameters (featureFlagOtelTraceSampling, openTelemetryTracingSampling BSON doc, opentelemetryTraceDirectory) and NDJSON export format. Fixed the prose test to assert operation-span parentage: the driver injects the traceparent from the OperationContext's operation span (per design 3.1), so the server span is a sibling of the client command span.

…crometer-tracing

…llMarked

…llMarked contract)

…bda)

…coded-bytes dependency)

…try section

…runbook

…e JAVA ticket)

…en, limitation noted)

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

driver-core/src/main/com/mongodb/internal/operation/ServerVersionHelper.java:38

LATEST_WIRE_VERSION is still set to the 8.0 value even though this PR introduces NINE_DOT_ZERO_WIRE_VERSION. This makes the constant name misleading and causes unit tests that use LATEST_WIRE_VERSION to stop representing the newest supported wire version.

    // Server 9.0 (WIRE_VERSION_90 = 29). Minimum wire version for the OP_MSG telemetry
    // section (OTel trace-context propagation, DRIVERS-3454).
    public static final int NINE_DOT_ZERO_WIRE_VERSION = 29;
    public static final int LATEST_WIRE_VERSION = EIGHT_DOT_ZERO_WIRE_VERSION;

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Comments suppressed due to low confidence (1)

driver-core/src/main/com/mongodb/internal/operation/ServerVersionHelper.java:37

LATEST_WIRE_VERSION is now inconsistent with the newly introduced NINE_DOT_ZERO_WIRE_VERSION constant. Keeping it pinned to 8.0 makes test utilities and any future uses of LATEST_WIRE_VERSION silently exclude wire version 29 features (like OP_MSG telemetry).

    // Server 9.0 (WIRE_VERSION_90 = 29). Minimum wire version for the OP_MSG telemetry
    // section (OTel trace-context propagation, DRIVERS-3454).
    public static final int NINE_DOT_ZERO_WIRE_VERSION = 29;
    public static final int LATEST_WIRE_VERSION = EIGHT_DOT_ZERO_WIRE_VERSION;

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated no new comments.

Comments suppressed due to low confidence (2)

driver-core/src/main/com/mongodb/internal/operation/ServerVersionHelper.java:38

LATEST_WIRE_VERSION is now inconsistent with the newly added NINE_DOT_ZERO_WIRE_VERSION constant (the file comments call 9.0 / wire version 29 the minimum for telemetry support, but LATEST_WIRE_VERSION still points at 8.0 / 25). This makes the meaning of “latest” ambiguous for test setups that import LATEST_WIRE_VERSION.

Consider either updating LATEST_WIRE_VERSION to NINE_DOT_ZERO_WIRE_VERSION, or adjusting naming/docs to clarify why 8.0 remains the “latest” for the driver/test suite.

    // Server 9.0 (WIRE_VERSION_90 = 29). Minimum wire version for the OP_MSG telemetry
    // section (OTel trace-context propagation, DRIVERS-3454).
    public static final int NINE_DOT_ZERO_WIRE_VERSION = 29;
    public static final int LATEST_WIRE_VERSION = EIGHT_DOT_ZERO_WIRE_VERSION;

driver-core/src/main/com/mongodb/internal/connection/CommandMessage.java:221

Javadoc has an unbalanced parenthesis: “(first key of the command document.” should be “(first key of the command document).”

     * The command name (first key of the command document. Available before {@code encode()} runs, unlike

Copilot

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 1 comment.

Copilot

Pull request overview

Copilot reviewed 13 out of 13 changed files in this pull request and generated no new comments.

rozza

Looks good - have a few comments / suggestions / questions and nits.

A wider question is should we bump micrometer in versions? Currently we have:

micrometer-tracing = "1.6.0-M3" # This version has a fix for https://github.com/micrometer-metrics/tracing/issues/1092
micrometer-observation-bom = "1.15.4"

Should we just bump to 1.16.6 (latest 1.16.6) or 1.17.0 (latest)?

rozza · 2026-07-28T09:20:57Z

+     * key, since encoding only appends fields.
+     */
+    public String getCommandName() {
+        return command.getFirstKey();


Is it worth caching this in the constructor? I don't think theres a path where we don't call it?

rozza · 2026-07-28T09:24:54Z

+     * on the classpath (e.g. metrics/logging-only observability setups). Guard any use of {@code io.micrometer.tracing.*}
+     * types with this flag so that such setups never trigger a {@link NoClassDefFoundError} or {@link LinkageError}.
+     */
+    static final boolean MICROMETER_TRACING_ON_CLASSPATH = isMicrometerTracingOnClasspath();


Nit: Lets annotation this as visible for testing otherwise private:

@VisibleForTesting(otherwise = AccessModifier.PRIVATE)

I would have suggested but imports are annoying with suggestions.

rozza · 2026-07-28T09:33:23Z

+     * when {@code micrometer-tracing} is absent from the classpath.
+     */
+    private static final class MicrometerTracingSupport {
+        @Nullable


Suggest adding a simple javadoc to help grok this.

Suggested change

@Nullable

/**

* Computes the W3C {@code traceparent} for {@code observation}'s active span.

*

* @param observation the observation whose span supplies the trace context

* @return the {@code traceparent} as specified by {@link TraceContext#traceParent()}, or {@code null} when

* {@code observation} has no span or its trace/span ids are missing, zero, or malformed

*/

@Nullable

rozza · 2026-07-28T09:34:23Z

+     * The 55-char W3C {@code traceparent} string for this context
+     * ({@code 00-<32 hex traceId>-<16 hex spanId>-<flags>}; flags {@code 01} sampled / {@code 00} unsampled),
+     * or {@code null} when there is no valid context (no-op span, missing/zero/malformed ids).
+     * Never includes tracestate.


rozza · 2026-07-28T09:35:13Z

+            BsonInt64 getMoreCursorId = message.getGetMoreCursorId();
+            if (getMoreCursorId != null) {
+                mongodbContext.setCursorId(getMoreCursorId.longValue());


See previous suggestion for using a Long

Suggested change

BsonInt64 getMoreCursorId = message.getGetMoreCursorId();

if (getMoreCursorId != null) {

mongodbContext.setCursorId(getMoreCursorId.longValue());

Long getMoreCursorId = message.getGetMoreCursorId();

if (getMoreCursorId != null) {

mongodbContext.setCursorId(getMoreCursorId);

rozza · 2026-07-28T09:37:40Z

+    /**
+     * Walks the OP_MSG sections skip the type-0 body document, then for each subsequent section read the kind byte;
+     * for kind 1 (document sequence) skip past the {@code int32} section size, and for kind 3 (telemetry) decode and
+     * return the BSON document payload. Returns {@code null} if no kind-3 section is found.


Nit: Slightly easier to read

Suggested change

/**

* Walks the OP_MSG sections skip the type-0 body document, then for each subsequent section read the kind byte;

* for kind 1 (document sequence) skip past the {@code int32} section size, and for kind 3 (telemetry) decode and

* return the BSON document payload. Returns {@code null} if no kind-3 section is found.

/**

* Walks the OP_MSG sections: skips the type-0 body document, then reads the kind byte of each subsequent

* section. For kind 1 (document sequence) it skips past the {@code int32} section size; for kind 3 (telemetry)

* it decodes and returns the BSON document payload. Returns {@code null} if no kind-3 section is found.

*/

rozza · 2026-07-28T09:44:06Z

+     */
+    public void encode(final ByteBufferBsonOutput bsonOutput, final OperationContext operationContext,
+            @Nullable final Span commandSpan) {
+        this.commandSpanForEncoding = commandSpan;


Question: This feels like an antipattern - global state that is safe with the try finally but its a bit hidden.

Would it be clearer if this was threaded into encode as a parameter instead?

rozza · 2026-07-28T09:48:37Z

+    public BsonInt64 getGetMoreCursorId() {
+        BsonValue value = command.get("getMore");
+        return value instanceof BsonInt64 ? (BsonInt64) value : null;
+    }


I'd recommend changing to a Long as that mirrors the getCursorId methods in the code base. See: CommandCursorResult setting of cursorId

Suggested change

public BsonInt64 getGetMoreCursorId() {

BsonValue value = command.get("getMore");

return value instanceof BsonInt64 ? (BsonInt64) value : null;

}

public Long getGetMoreCursorId() {

BsonValue value = command.get("getMore");

return value.isNumber() ? value.asNumber().longValue() : null;

}

I also made a suggestion below to change getGetMoreCursorId use.

rozza · 2026-07-28T09:57:37Z

@@ -441,20 +439,19 @@ public boolean reauthenticationIsTriggered(@Nullable final Throwable t) {
    @Nullable
    private <T> T sendAndReceiveInternal(final CommandMessage message, final Decoder<T> decoder,


I got this feedback from Claude:

🟡 `[important]` — `InternalStreamConnection.java`: test gap The connection-level rewiring introduced here has no direct coverage. The new CommandMessage/traceparent tests exercise encode(...) in isolation and never construct an InternalStreamConnection, and the existing InternalStreamConnectionSpecification predates these changes. Not covered: 1. That the sync and async paths actually call encode(bsonOutput, operationContext, tracingSpan) (the 3-arg overload) with the span from createTracingSpan. A regression to the 2-arg overload or null would pass all current tests yet silently stop emitting the telemetry section. 2. The reorder — span created before encode. 3. Exactly-once semantics under a send failure: the widened catch (Throwable) firing sendFailedEvent + span.error()/closeScope()/end() once each, without the receive path double-firing. 4. The intentional sync/async asymmetry in the catch (async does no closeScope() / direct sendFailedEvent). Notes: • End-to-end server acceptance of the kind-3 section is 9.0-gated and reasonably deferred. • The wiring / ordering / failure paths are unit-testable with the existing TestInternalConnection + Spock harness and a mocked TracingManager/Span. • Per AGENTS.md ("every code change must include tests"), this should be covered on a safety-critical send path.

It may be worth auto migrating InternalStreamConnectionSpecification.groovy to Junit 5 and adding extra coverage tests.

…ace-context prose tests Adds an --otel/OTEL=1 flag to drivers_orchestration.py that starts every mongod/mongos with the MongoDB 9.0+ OpenTelemetry file-exporter setParameters (per-port trace directories under $DRIVERS_TOOLS/otel, samplingFactor 1.0 with raised token buckets for both default and external sampling per the server IDL), exports OTEL_TRACE_DIR via mo-expansion.sh/.yml, fails fast on incompatible modes (version < 9.0, DOCKER_RUNNING, --local-atlas, --mongodb-runner), and wipes the trace directory between executions. Includes stdlib-unittest coverage, README documentation with driver integration guidance, and the design/plan docs. Prerequisite for the prose test in mongodb/specifications#1966; reference implementation mongodb/mongo-java-driver#2022.

…th/monitoring spans) Runs against an OTel-enabled 9.0+ deployment (drivers-evergreen-tools OTEL=1, DRIVERS-3605); gated on -Dorg.mongodb.test.otel.trace.dir. Parses the exported OTLP NDJSON batches as JSON; fails explicitly when no span files appear (non-OTel build) instead of a generic timeout.

Threads OTEL through start-mongo-orchestration (REQUIRE_API_VERSION pattern), adds otel-tracing-prose-test-task (temporary fork checkout until mongodb-labs/drivers-evergreen-tools#837 merges), a latest/standalone variant, and otel trace artifact upload in post.

Test 5 uses insertOne and asserts exactly one parented server span; Test 6 asserts shared traceId and exactly two per-attempt server spans (prose updated to the retryable-read variant so it runs on all topologies); the CI variant enables auth so Test 7 exercises the sasl commands as written.

…ERS-3605 merged upstream)

nhachicha added 30 commits June 1, 2026 18:51

DRIVERS-3454: design for OTel trace-context propagation over OP_MSG

f991058

Spec proposal (OP_MSG section kind 3, W3C traceparent, hello tracingSupport negotiation) plus driver-sync POC design to validate it.

DRIVERS-3454: implementation plan for OP_MSG trace-context POC (drive…

71944e9

…r-sync)

POC WIP

c23e2c9

DRIVERS-3454: design for reference implementation (BSON telemetry sec…

b7def2e

…tion, wire-version gate)

DRIVERS-3454: implementation plan for reference implementation

10a7e48

Merge remote-tracking branch 'origin/main' into nabil_otel_context

22460aa

DRIVERS-3454: add NINE_DOT_ZERO_WIRE_VERSION (29) constant

eb27cfa

DRIVERS-3454: guarantee traceParent() is server-valid; propagate unsa…

44fd4ca

…mpled with flags 00

DRIVERS-3454: write BSON telemetry section gated on maxWireVersion >= 29

9bd5c65

DRIVERS-3454: remove hello-capability plumbing and test toggle (wire-…

9931e89

…version gate supersedes)

DRIVERS-3454: prose-test definitions for trace-context propagation

d943d9e

DRIVERS-3454: e2e server-span linkage prose test (OTLP file exporter)

0176242

DRIVERS-3454: mark POC design superseded

321b7c3

DRIVERS-3454: guard traceparent production against absent optional mi…

c36118c

…crometer-tracing

DRIVERS-3454: drop dead null-check on Span.context() (non-null by @Nu…

9bf631a

…llMarked contract)

DRIVERS-3454: IDE polish (suppress always-inverted hint, EMPTY as lam…

f067ee1

…bda)

DRIVERS-3454: design for minimal spec change (OP_MSG + OTel specs)

9d7cfd6

DRIVERS-3454: plan for minimal spec change

4c3715f

DRIVERS-3454: design for command-span trace propagation (Option A)

fe04efd

DRIVERS-3454: plan for command-span trace propagation

346a959

DRIVERS-3454: createTracingSpan reads the raw command document (no en…

e295fa0

…coded-bytes dependency)

DRIVERS-3454: encode overload carries the command span for the teleme…

6db604c

…try section

DRIVERS-3454: create command span before encode; propagate its context

7352282

DRIVERS-3454: e2e + prose docs assert command-span parentage

b3e5358

DRIVERS-3454: note command-span amendment in reference-impl design

dc9fa67

DRIVERS-3454: fix stale mechanism comment in e2e test

80b0c2f

DRIVERS-3454: correct traceparent-mechanism wording in prose doc and …

4c5e5dc

…runbook

DRIVERS-3454: document pre-existing sync-path span-leak window (futur…

8cade28

…e JAVA ticket)

DRIVERS-3454: record prose-test decision (deferred; interop test chos…

7369f1f

…en, limitation noted)

Copilot started reviewing on behalf of nhachicha July 23, 2026 12:01 View session

Copilot AI reviewed Jul 23, 2026

View reviewed changes

Comment thread gradle/libs.versions.toml Outdated

nhachicha requested a review from Copilot July 23, 2026 12:09

Copilot started reviewing on behalf of nhachicha July 23, 2026 12:10 View session

Copilot AI reviewed Jul 23, 2026

View reviewed changes

Comment thread driver-core/src/main/com/mongodb/internal/connection/CommandMessage.java

nhachicha requested a review from Copilot July 23, 2026 12:16

Copilot started reviewing on behalf of nhachicha July 23, 2026 12:16 View session

Copilot AI reviewed Jul 23, 2026

View reviewed changes

Comment thread ...e/src/test/unit/com/mongodb/internal/observability/micrometer/MicrometerTraceParentTest.java

nhachicha requested a review from Copilot July 23, 2026 12:26

Copilot started reviewing on behalf of nhachicha July 23, 2026 12:26 View session

Copilot AI reviewed Jul 23, 2026

View reviewed changes

nhachicha requested a review from Copilot July 23, 2026 12:33

Copilot started reviewing on behalf of nhachicha July 23, 2026 12:34 View session

Copilot AI reviewed Jul 23, 2026

View reviewed changes

Comment thread driver-core/src/main/com/mongodb/internal/connection/CommandMessage.java

nhachicha added 2 commits July 23, 2026 15:26

Bump to wire protocol 29 and Copilot feedback

2ce9337

review feedback

73cf17b

nhachicha requested a review from Copilot July 23, 2026 14:44

Copilot started reviewing on behalf of nhachicha July 23, 2026 14:44 View session

Copilot AI reviewed Jul 23, 2026

View reviewed changes

nhachicha requested a review from rozza July 23, 2026 14:50

nhachicha marked this pull request as ready for review July 23, 2026 14:50

nhachicha requested a review from a team as a code owner July 23, 2026 14:50

rozza requested changes Jul 28, 2026

View reviewed changes

nhachicha mentioned this pull request Sep 3, 2026

DRIVERS-3605: add opt-in OTel span-export server configuration for trace-context prose tests mongodb-labs/drivers-evergreen-tools#837

Merged

6 tasks

nhachicha added 4 commits September 9, 2026 14:21

JAVA-6222: drop temporary drivers-evergreen-tools fork checkout (DRIV…

be1d4ba

…ERS-3605 merged upstream)

-        @Nullable
+          /**
+           * Computes the W3C {@code traceparent} for {@code observation}'s active span.
+           *
+           * @param observation the observation whose span supplies the trace context
+           * @return the {@code traceparent} as specified by {@link TraceContext#traceParent()}, or {@code null} when
+           *         {@code observation} has no span or its trace/span ids are missing, zero, or malformed
+           */
+        @Nullable

		@@ -441,20 +439,19 @@ public boolean reauthenticationIsTriggered(@Nullable final Throwable t) {
		@Nullable
		private <T> T sendAndReceiveInternal(final CommandMessage message, final Decoder<T> decoder,

Conversation

nhachicha commented Jul 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

rozza left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

nhachicha commented Jul 23, 2026 •

edited

Loading