feat(data-pipeline): add meta struct blob setter - #2303
Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 7 commits intoAug 4, 2026
Conversation
Expose a C setter that copies an opaque binary value into a span's `meta_struct` map. Native tracers can now supply pre-encoded MessagePack without transferring ownership or requiring Rust to interpret it. Reject null span handles and non-UTF-8 keys. Reusing a key replaces its value, and the copied bytes remain owned by the span after the call.
BenchmarksComparisonCandidateCandidate benchmark detailsBaselineBaseline benchmark details |
🎉 All green!🧪 All tests passed 🎯 Code Coverage (details) 🔗 Commit SHA: 220d6f8 | Docs | Datadog PR Page | Give us feedback! |
Contributor
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
Member
Author
|
Draft dd-trace-rb example consumer: DataDog/dd-trace-rb#6130. |
lloeki
force-pushed
the
lloeki/native-meta-struct-m1
branch
from
July 30, 2026 12:02
69926bd to
27ad1af
Compare
This was referenced Jul 30, 2026
ekump
reviewed
Aug 2, 2026
| }; | ||
| span.0 | ||
| .meta_struct | ||
| .insert(key, Bytes::copy_from_slice(value.as_bytes())); |
Collaborator
There was a problem hiding this comment.
value.as_bytes() can panic when there is invalid input (example: null pointer, non-zero length). You can use try_as_bytes instead to get a more meaningful error.
ekump
reviewed
Aug 2, 2026
| /// The `key` and opaque binary `value` are copied into the span. The value is | ||
| /// not interpreted or validated as MessagePack. | ||
| /// | ||
| /// Repeating a `key` appends a new entry rather than replacing the previous one. The last value |
Collaborator
There was a problem hiding this comment.
This exposes an internal implementation detail, so maybe shouldn't be here? I figure it's worth it since it could have performance implications for meta_struct
Member
Author
There was a problem hiding this comment.
Your comment was confusing to me until I read the following line:
/// written for a key is the one read back and the one serialized
And with that, agreed: it just appears to behave as a regular setter to the API consumer.
ekump
approved these changes
Aug 2, 2026
ekump
left a comment
ekump
left a comment
Collaborator
There was a problem hiding this comment.
LGTM but someone else should review since I also pushed commits
hoolioh
approved these changes
Aug 3, 2026
hoolioh
left a comment
hoolioh
left a comment
Contributor
There was a problem hiding this comment.
LGTM, a little too much verbosity regarding comments though.
gh-worker-dd-mergequeue-cf854d Bot
pushed a commit
that referenced
this pull request
Aug 4, 2026
AI-generated code disclosure: this PR was implemented with substantial AI assistance and manually reviewed and validated. # What does this PR do? Ensures the combined libdatadog profiling artifact enables `catch_panic` for data-pipeline FFI entry points. It also adds a packaged C regression that triggers capacity overflow and verifies that the FFI returns an error instead of aborting the process. # Motivation The standalone data-pipeline FFI crate enables panic containment by default, but the combined profiling artifact disables dependency defaults and did not re-enable it. This closes [APMSP-3830](https://datadoghq.atlassian.net/browse/APMSP-3830) before the native trace exporter payload APIs ship together. # Additional Notes **Stack position: L0, current PR: [DataDog/libdatadog PR 2302](#2302 These open L0-L5 changes are not included in [libdatadog v38.0.0](https://github.com/DataDog/libdatadog/releases/tag/v38.0.0). They are planned to ship together in a single libdatadog v39 release, with the common dd-trace-rb uptake tracked by [R3 DataDog/dd-trace-rb PR 6143](DataDog/dd-trace-rb#6143). Complete expected libdatadog review and merge order for that single v39 release: | Level | libdatadog PR | Matching dd-trace-rb consumer | | --- | --- | --- | | **L0 (current)** | **[DataDog/libdatadog PR 2302](#2302 | Release prerequisite; no payload API consumer | | L1 | [DataDog/libdatadog PR 2303](#2303) | [R4 DataDog/dd-trace-rb PR 6130](DataDog/dd-trace-rb#6130) | | L2 | [DataDog/libdatadog PR 2305](#2305) | [R5 DataDog/dd-trace-rb PR 6129](DataDog/dd-trace-rb#6129) | | L3 | [DataDog/libdatadog PR 2301](#2301) | [R6 DataDog/dd-trace-rb PR 6134](DataDog/dd-trace-rb#6134) | | L4 | [DataDog/libdatadog PR 2304](#2304) | [R7 DataDog/dd-trace-rb PR 6132](DataDog/dd-trace-rb#6132) | | L5 | [DataDog/libdatadog PR 2300](#2300) | [R8 DataDog/dd-trace-rb PR 6133](DataDog/dd-trace-rb#6133) | L0-L3 are independent and can be reviewed in parallel. Merging them in the listed order is still recommended because it reduces integration and release-branch conflicts, and lands panic containment before the new payload surfaces. L4 is stacked on L1, and L5 is stacked on L4. The matching payload consumers are [R4 DataDog/dd-trace-rb PR 6130](DataDog/dd-trace-rb#6130), [R5 DataDog/dd-trace-rb PR 6129](DataDog/dd-trace-rb#6129), [R6 DataDog/dd-trace-rb PR 6134](DataDog/dd-trace-rb#6134), [R7 DataDog/dd-trace-rb PR 6132](DataDog/dd-trace-rb#6132), and [R8 DataDog/dd-trace-rb PR 6133](DataDog/dd-trace-rb#6133). All depend through the common v39 uptake in [R3 DataDog/dd-trace-rb PR 6143](DataDog/dd-trace-rb#6143). The downstream review and merge order after R3 is R4, R5, R6, R7, then R8; R7 also depends on R4, and R8 depends on R7. [DataDog/libdatadog-rb v37 PR 65](DataDog/libdatadog-rb#65) and [DataDog/libdatadog-rb v38 PR 66](DataDog/libdatadog-rb#66) are separate ordered prerequisite releases only for [R2 DataDog/dd-trace-rb PR 6131](DataDog/dd-trace-rb#6131) and its `ForkSafeRuntime` uptake. They do not contain the open L0-L5 changes or payload APIs. R3 is stacked on R2, but R3, rather than either prerequisite package, is the common v39 payload-API uptake. L0 tracks [APMSP-3830](https://datadoghq.atlassian.net/browse/APMSP-3830); L1-L5 implement the payload work tracked by [APMSP-3219](https://datadoghq.atlassian.net/browse/APMSP-3219). # How to test the change? Prior local validation completed successfully: - `nix develop -c cargo test -p libdd-profiling-ffi --features data-pipeline-ffi` passed the combined profiling/data-pipeline FFI suite: 57 tests. - `nix develop -c cargo test -p libdd-data-pipeline-ffi --features catch_panic trace_chunks_new_with_overflow_capacity_returns_panic_error` passed. - `nix develop -c cargo test -p libdd-data-pipeline-ffi --features catch_panic begin_chunk_with_overflow_capacity_returns_panic_error` passed. - `nix develop -c cargo ffi-test --filter trace_exporter_panic_containment` built and passed the packaged C panic-containment regression. - `nix develop .#nightly --command cargo fmt --all -- --check` passed. - `nix develop -c cargo clippy -p libdd-profiling-ffi --all-targets --features data-pipeline-ffi -- -D warnings` passed. [APMSP-3830]: https://datadoghq.atlassian.net/browse/APMSP-3830?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [APMSP-3830]: https://datadoghq.atlassian.net/browse/APMSP-3830?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ Co-authored-by: ekump <edmund.kump@datadoghq.com> Co-authored-by: hoolioh <julio.gonzalez@datadoghq.com>
Member
Author
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
|
gh-worker-dd-devflow-36fce6
Bot
added
mergequeue-status: queued
mergequeue-status: in_progress
and removed
mergequeue-status: queued
labels
gh-worker-dd-devflow-36fce6
Bot
added
mergequeue-status: done
and removed
mergequeue-status: in_progress
labels
gh-worker-dd-mergequeue-cf854d
Bot
deleted the
lloeki/native-meta-struct-m1
branch
August 4, 2026 11:48
hoolioh
added a commit
that referenced
this pull request
Aug 5, 2026
AI-generated code disclosure: this PR was implemented with substantial AI assistance and manually reviewed and validated. # What does this PR do? Adds owned span-event construction and atomic attachment through the data-pipeline FFI. It supports typed scalar attributes, homogeneous typed arrays, empty arrays, event ordering, and UTF-8 validation. # Motivation The Rust trace model supports span events, but native exporter consumers cannot populate them. This fills the span-event portion of the payload parity work tracked by [APMSP-3219](https://datadoghq.atlassian.net/browse/APMSP-3219). # Additional Notes **Stack position: L3, current PR: [DataDog/libdatadog PR 2301](#2301 These open L0-L5 changes are not included in [libdatadog v38.0.0](https://github.com/DataDog/libdatadog/releases/tag/v38.0.0). They are planned to ship together in a single libdatadog v39 release, with the common dd-trace-rb uptake tracked by [R3 DataDog/dd-trace-rb PR 6143](DataDog/dd-trace-rb#6143). Complete expected libdatadog review and merge order for that single v39 release: | Level | libdatadog PR | Matching dd-trace-rb consumer | | --- | --- | --- | | L0 | [DataDog/libdatadog PR 2302](#2302) | Release prerequisite; no payload API consumer | | L1 | [DataDog/libdatadog PR 2303](#2303) | [R4 DataDog/dd-trace-rb PR 6130](DataDog/dd-trace-rb#6130) | | L2 | [DataDog/libdatadog PR 2305](#2305) | [R5 DataDog/dd-trace-rb PR 6129](DataDog/dd-trace-rb#6129) | | **L3 (current)** | **[DataDog/libdatadog PR 2301](#2301 | **[R6 DataDog/dd-trace-rb PR 6134](DataDog/dd-trace-rb#6134 | | L4 | [DataDog/libdatadog PR 2304](#2304) | [R7 DataDog/dd-trace-rb PR 6132](DataDog/dd-trace-rb#6132) | | L5 | [DataDog/libdatadog PR 2300](#2300) | [R8 DataDog/dd-trace-rb PR 6133](DataDog/dd-trace-rb#6133) | L0-L3 are independent and can be reviewed in parallel. Merging them in the listed order is still recommended because it reduces integration and release-branch conflicts, and lands panic containment before the new payload surfaces. L4 is stacked on L1, and L5 is stacked on L4. The matching payload consumers are [R4 DataDog/dd-trace-rb PR 6130](DataDog/dd-trace-rb#6130), [R5 DataDog/dd-trace-rb PR 6129](DataDog/dd-trace-rb#6129), [R6 DataDog/dd-trace-rb PR 6134](DataDog/dd-trace-rb#6134), [R7 DataDog/dd-trace-rb PR 6132](DataDog/dd-trace-rb#6132), and [R8 DataDog/dd-trace-rb PR 6133](DataDog/dd-trace-rb#6133). All depend through the common v39 uptake in [R3 DataDog/dd-trace-rb PR 6143](DataDog/dd-trace-rb#6143). The downstream review and merge order after R3 is R4, R5, R6, R7, then R8; R7 also depends on R4, and R8 depends on R7. [DataDog/libdatadog-rb v37 PR 65](DataDog/libdatadog-rb#65) and [DataDog/libdatadog-rb v38 PR 66](DataDog/libdatadog-rb#66) are separate ordered prerequisite releases only for [R2 DataDog/dd-trace-rb PR 6131](DataDog/dd-trace-rb#6131) and its `ForkSafeRuntime` uptake. They do not contain the open L0-L5 changes or payload APIs. R3 is stacked on R2, but R3, rather than either prerequisite package, is the common v39 payload-API uptake. L0 tracks [APMSP-3830](https://datadoghq.atlassian.net/browse/APMSP-3830); L1-L5 implement the payload work tracked by [APMSP-3219](https://datadoghq.atlassian.net/browse/APMSP-3219). # How to test the change? Prior local validation completed successfully: - `nix develop --command cargo test -p libdd-data-pipeline-ffi` passed 58 tests, including scalar values, homogeneous typed arrays, empty arrays, ordering, ownership, and atomic failure coverage. - `nix develop --command cargo clippy -p libdd-data-pipeline-ffi --all-targets -- -D warnings` passed. - `nix develop .#nightly --command cargo fmt --all -- --check` passed. - `nix develop --command env LIBDATADOG_VENDOR_OVERRIDE=/tmp/opencode/native-span-events-ruby bundle exec rspec spec/datadog/tracing/transport/native/tracer_span_spec.rb spec/datadog/tracing/transport/native/send_traces_spec.rb spec/datadog/tracing/transport/native/transport_spec.rb spec/datadog/tracing/transport/traces_spec.rb spec/datadog/tracing/span_event_spec.rb spec/datadog/tracing/span_operation_spec.rb` passed 123 native and producer examples, including native wire output and legacy JSON fallback. - Targeted Standard and Steep checks for the native transport and span-event paths passed. [APMSP-3219]: https://datadoghq.atlassian.net/browse/APMSP-3219?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [APMSP-3830]: https://datadoghq.atlassian.net/browse/APMSP-3830?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: Julio <julio.gonzalez@datadoghq.com> Co-authored-by: Edmund Kump <edmund.kump@datadoghq.com> Co-authored-by: Julio Gonzalez <107922352+hoolioh@users.noreply.github.com>
hoolioh
added a commit
that referenced
this pull request
Aug 5, 2026
AI-generated code disclosure: this PR was implemented with substantial AI assistance and manually reviewed and validated. # What does this PR do? Adds an atomic batched FFI for complete span links, including 128-bit trace IDs, span IDs, flags, tracestate, attributes, and ordering. All borrowed input is validated and copied before insertion. # Motivation The Rust trace model supports span links, but native exporter consumers cannot populate it. This fills the span-link portion of the payload parity work tracked by [APMSP-3219](https://datadoghq.atlassian.net/browse/APMSP-3219). # Additional Notes `dropped_attributes_count` is intentionally deferred by additive commit [961044e](961044e). Neither the Agent v0.4 nor indexed V1 span-link schema defines the field, and current Datadog tracers do not produce non-zero counts. The original implementation remains in the preceding commit and can be restored by reverting the deferral commit. **Stack position: L2, current PR: [DataDog/libdatadog PR 2305](#2305 These open L0-L5 changes are not included in [libdatadog v38.0.0](https://github.com/DataDog/libdatadog/releases/tag/v38.0.0). They are planned to ship together in a single libdatadog v39 release, with the common dd-trace-rb uptake tracked by [R3 DataDog/dd-trace-rb PR 6143](DataDog/dd-trace-rb#6143). Complete expected libdatadog review and merge order for that single v39 release: | Level | libdatadog PR | Matching dd-trace-rb consumer | | --- | --- | --- | | L0 | [DataDog/libdatadog PR 2302](#2302) | Release prerequisite; no payload API consumer | | L1 | [DataDog/libdatadog PR 2303](#2303) | [R4 DataDog/dd-trace-rb PR 6130](DataDog/dd-trace-rb#6130) | | **L2 (current)** | **[DataDog/libdatadog PR 2305](#2305 | **[R5 DataDog/dd-trace-rb PR 6129](DataDog/dd-trace-rb#6129 | | L3 | [DataDog/libdatadog PR 2301](#2301) | [R6 DataDog/dd-trace-rb PR 6134](DataDog/dd-trace-rb#6134) | | L4 | [DataDog/libdatadog PR 2304](#2304) | [R7 DataDog/dd-trace-rb PR 6132](DataDog/dd-trace-rb#6132) | | L5 | [DataDog/libdatadog PR 2300](#2300) | [R8 DataDog/dd-trace-rb PR 6133](DataDog/dd-trace-rb#6133) | L0-L3 are independent and can be reviewed in parallel. Merging them in the listed order is still recommended because it reduces integration and release-branch conflicts, and lands panic containment before the new payload surfaces. L4 is stacked on L1, and L5 is stacked on L4. The matching payload consumers are [R4 DataDog/dd-trace-rb PR 6130](DataDog/dd-trace-rb#6130), [R5 DataDog/dd-trace-rb PR 6129](DataDog/dd-trace-rb#6129), [R6 DataDog/dd-trace-rb PR 6134](DataDog/dd-trace-rb#6134), [R7 DataDog/dd-trace-rb PR 6132](DataDog/dd-trace-rb#6132), and [R8 DataDog/dd-trace-rb PR 6133](DataDog/dd-trace-rb#6133). All depend through the common v39 uptake in [R3 DataDog/dd-trace-rb PR 6143](DataDog/dd-trace-rb#6143). The downstream review and merge order after R3 is R4, R5, R6, R7, then R8; R7 also depends on R4, and R8 depends on R7. [DataDog/libdatadog-rb v37 PR 65](DataDog/libdatadog-rb#65) and [DataDog/libdatadog-rb v38 PR 66](DataDog/libdatadog-rb#66) are separate ordered prerequisite releases only for [R2 DataDog/dd-trace-rb PR 6131](DataDog/dd-trace-rb#6131) and its `ForkSafeRuntime` uptake. They do not contain the open L0-L5 changes or payload APIs. R3 is stacked on R2, but R3, rather than either prerequisite package, is the common v39 payload-API uptake. L0 tracks [APMSP-3830](https://datadoghq.atlassian.net/browse/APMSP-3830); L1-L5 implement the payload work tracked by [APMSP-3219](https://datadoghq.atlassian.net/browse/APMSP-3219). # How to test the change? Prior local validation completed successfully: - `nix develop -c cargo nextest run -p libdd-data-pipeline-ffi` passed 59 tests, including atomic invalid-UTF-8 failure, ordering, and ownership coverage. - `nix develop -c cargo test -p libdd-trace-utils` passed the unit, integration, and doc-test coverage for all affected encoders and decoders. - `nix develop -c cargo clippy -p libdd-trace-utils -p libdd-data-pipeline-ffi --all-targets -- -D warnings` passed. - `nix develop .#nightly --command cargo fmt --all -- --check` passed. - `nix develop --command bundle exec rspec spec/datadog/tracing/transport/native/tracer_span_spec.rb spec/datadog/tracing/transport/native/transport_spec.rb spec/datadog/tracing/transport/native/conformance_spec.rb` passed 119 native transport examples against the locally built artifact. - `nix develop --command bundle exec rspec spec/datadog/tracing/span_link_spec.rb spec/datadog/tracing/distributed/propagation_spec.rb` passed 69 distributed-propagation examples with 1 existing pending example. - `nix develop --command env BUNDLE_GEMFILE=gemfiles/ruby_4.0_opentelemetry.gemfile bundle exec rspec spec/datadog/opentelemetry_spec.rb` passed 164 OpenTelemetry examples. - Full Standard and Steep checks passed on the matching consumer branch. [APMSP-3219]: https://datadoghq.atlassian.net/browse/APMSP-3219?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [APMSP-3830]: https://datadoghq.atlassian.net/browse/APMSP-3830?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ --------- Co-authored-by: Edmund Kump <edmund.kump@datadoghq.com> Co-authored-by: Julio <julio.gonzalez@datadoghq.com> Co-authored-by: Julio Gonzalez <107922352+hoolioh@users.noreply.github.com>
gh-worker-dd-mergequeue-cf854d Bot
pushed a commit
that referenced
this pull request
Aug 6, 2026
AI-generated code disclosure: this PR was implemented with substantial AI assistance and manually reviewed and validated. # What does this PR do? Introduces a validated, language-neutral structured-value token ABI and standalone MessagePack encoder. It supports nil, booleans, signed and unsigned integers, floats, UTF-8 strings, binary strings, arrays, and maps, with depth and shape validation. # Motivation The opaque blob API in [DataDog/libdatadog PR 2303](#2303) restores transitional parity but still requires every consumer to encode MessagePack. This stage moves validated structured-value encoding into libdatadog as part of [APMSP-3219](https://datadoghq.atlassian.net/browse/APMSP-3219). # Additional Notes **Stack position: L4, current PR: [DataDog/libdatadog PR 2304](#2304), stacked on L1 [DataDog/libdatadog PR 2303](#2303 These open L0-L5 changes are not included in [libdatadog v38.0.0](https://github.com/DataDog/libdatadog/releases/tag/v38.0.0). They are planned to ship together in a single libdatadog v39 release, with the common dd-trace-rb uptake tracked by [R3 DataDog/dd-trace-rb PR 6143](DataDog/dd-trace-rb#6143). Complete expected libdatadog review and merge order for that single v39 release: | Level | libdatadog PR | Matching dd-trace-rb consumer | | --- | --- | --- | | L0 | [DataDog/libdatadog PR 2302](#2302) | Release prerequisite; no payload API consumer | | L1 | [DataDog/libdatadog PR 2303](#2303) | [R4 DataDog/dd-trace-rb PR 6130](DataDog/dd-trace-rb#6130) | | L2 | [DataDog/libdatadog PR 2305](#2305) | [R5 DataDog/dd-trace-rb PR 6129](DataDog/dd-trace-rb#6129) | | L3 | [DataDog/libdatadog PR 2301](#2301) | [R6 DataDog/dd-trace-rb PR 6134](DataDog/dd-trace-rb#6134) | | **L4 (current)** | **[DataDog/libdatadog PR 2304](#2304 | **[R7 DataDog/dd-trace-rb PR 6132](DataDog/dd-trace-rb#6132 | | L5 | [DataDog/libdatadog PR 2300](#2300) | [R8 DataDog/dd-trace-rb PR 6133](DataDog/dd-trace-rb#6133) | L0-L3 are independent and can be reviewed in parallel. Merging them in the listed order is still recommended because it reduces integration and release-branch conflicts, and lands panic containment before the new payload surfaces. L4 is stacked on L1, and L5 is stacked on L4. The matching payload consumers are [R4 DataDog/dd-trace-rb PR 6130](DataDog/dd-trace-rb#6130), [R5 DataDog/dd-trace-rb PR 6129](DataDog/dd-trace-rb#6129), [R6 DataDog/dd-trace-rb PR 6134](DataDog/dd-trace-rb#6134), [R7 DataDog/dd-trace-rb PR 6132](DataDog/dd-trace-rb#6132), and [R8 DataDog/dd-trace-rb PR 6133](DataDog/dd-trace-rb#6133). All depend through the common v39 uptake in [R3 DataDog/dd-trace-rb PR 6143](DataDog/dd-trace-rb#6143). The downstream review and merge order after R3 is R4, R5, R6, R7, then R8; R7 also depends on R4, and R8 depends on R7. [DataDog/libdatadog-rb v37 PR 65](DataDog/libdatadog-rb#65) and [DataDog/libdatadog-rb v38 PR 66](DataDog/libdatadog-rb#66) are separate ordered prerequisite releases only for [R2 DataDog/dd-trace-rb PR 6131](DataDog/dd-trace-rb#6131) and its `ForkSafeRuntime` uptake. They do not contain the open L0-L5 changes or payload APIs. R3 is stacked on R2, but R3, rather than either prerequisite package, is the common v39 payload-API uptake. This PR remains a draft because its review and final rebase depend on L1. Its matching consumer [R7 DataDog/dd-trace-rb PR 6132](DataDog/dd-trace-rb#6132) is also a draft pending this API, the v39 package, and [R3 DataDog/dd-trace-rb PR 6143](DataDog/dd-trace-rb#6143). L5 remains draft until this PR is ready. L0 tracks [APMSP-3830](https://datadoghq.atlassian.net/browse/APMSP-3830); L1-L5 implement the payload work tracked by [APMSP-3219](https://datadoghq.atlassian.net/browse/APMSP-3219). # How to test the change? Prior local validation completed successfully: - `nix develop -c cargo test -p libdd-data-pipeline-ffi structured_value` passed structured-value encoding, ownership, malformed-stream, shape, and depth-limit tests. - `nix develop -c cargo test -p libdd-data-pipeline-ffi` passed the complete data-pipeline FFI suite. - `nix develop -c cargo clippy -p libdd-data-pipeline-ffi --all-targets -- -D warnings` passed. - `nix develop .#nightly -c cargo fmt --all -- --check` passed. - The release artifact built successfully with `nix develop -c cargo run -p builder --bin release -- --out /tmp/opencode/libdatadog-m2/x86_64-linux`. - The matching consumer's native suite passed 122 examples, AppSec passed 807 examples, and AI Guard passed 125 examples against the local artifact. - Full Standard and Steep checks passed on the matching consumer branch. [APMSP-3219]: https://datadoghq.atlassian.net/browse/APMSP-3219?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [APMSP-3830]: https://datadoghq.atlassian.net/browse/APMSP-3830?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ Co-authored-by: ekump <edmund.kump@datadoghq.com> Co-authored-by: loic.nageleisen <loic.nageleisen@datadoghq.com>
gh-worker-dd-mergequeue-cf854d Bot
pushed a commit
that referenced
this pull request
Aug 6, 2026
AI-generated code disclosure: this PR was implemented with substantial AI assistance and manually reviewed and validated. # What does this PR do? Adds direct structured-value insertion into `TracerSpan.meta_struct`, reusing the validator and encoder from [DataDog/libdatadog PR 2304](#2304) while moving the encoded allocation directly into the span. The opaque blob and standalone encoder APIs remain available. # Motivation The standalone encoder still requires consumers to manage an intermediate encoded blob and an additional value copy. The direct setter provides the final efficient API for structured payload parity tracked by [APMSP-3219](https://datadoghq.atlassian.net/browse/APMSP-3219). # Additional Notes **Stack position: L5, current PR: [DataDog/libdatadog PR 2300](#2300), stacked on L4 [DataDog/libdatadog PR 2304](#2304 These open L0-L5 changes are not included in [libdatadog v38.0.0](https://github.com/DataDog/libdatadog/releases/tag/v38.0.0). They are planned to ship together in a single libdatadog v39 release, with the common dd-trace-rb uptake tracked by [R3 DataDog/dd-trace-rb PR 6143](DataDog/dd-trace-rb#6143). Complete expected libdatadog review and merge order for that single v39 release: | Level | libdatadog PR | Matching dd-trace-rb consumer | | --- | --- | --- | | L0 | [DataDog/libdatadog PR 2302](#2302) | Release prerequisite; no payload API consumer | | L1 | [DataDog/libdatadog PR 2303](#2303) | [R4 DataDog/dd-trace-rb PR 6130](DataDog/dd-trace-rb#6130) | | L2 | [DataDog/libdatadog PR 2305](#2305) | [R5 DataDog/dd-trace-rb PR 6129](DataDog/dd-trace-rb#6129) | | L3 | [DataDog/libdatadog PR 2301](#2301) | [R6 DataDog/dd-trace-rb PR 6134](DataDog/dd-trace-rb#6134) | | L4 | [DataDog/libdatadog PR 2304](#2304) | [R7 DataDog/dd-trace-rb PR 6132](DataDog/dd-trace-rb#6132) | | **L5 (current)** | **[DataDog/libdatadog PR 2300](#2300 | **[R8 DataDog/dd-trace-rb PR 6133](DataDog/dd-trace-rb#6133 | L0-L3 are independent and can be reviewed in parallel. Merging them in the listed order is still recommended because it reduces integration and release-branch conflicts, and lands panic containment before the new payload surfaces. L4 is stacked on L1, and L5 is stacked on L4. The matching payload consumers are [R4 DataDog/dd-trace-rb PR 6130](DataDog/dd-trace-rb#6130), [R5 DataDog/dd-trace-rb PR 6129](DataDog/dd-trace-rb#6129), [R6 DataDog/dd-trace-rb PR 6134](DataDog/dd-trace-rb#6134), [R7 DataDog/dd-trace-rb PR 6132](DataDog/dd-trace-rb#6132), and [R8 DataDog/dd-trace-rb PR 6133](DataDog/dd-trace-rb#6133). All depend through the common v39 uptake in [R3 DataDog/dd-trace-rb PR 6143](DataDog/dd-trace-rb#6143). The downstream review and merge order after R3 is R4, R5, R6, R7, then R8; R7 also depends on R4, and R8 depends on R7. [DataDog/libdatadog-rb v37 PR 65](DataDog/libdatadog-rb#65) and [DataDog/libdatadog-rb v38 PR 66](DataDog/libdatadog-rb#66) are separate ordered prerequisite releases only for [R2 DataDog/dd-trace-rb PR 6131](DataDog/dd-trace-rb#6131) and its `ForkSafeRuntime` uptake. They do not contain the open L0-L5 changes or payload APIs. R3 is stacked on R2, but R3, rather than either prerequisite package, is the common v39 payload-API uptake. This PR remains a draft because its review and final rebase depend on L4 and, transitively, L1. Its matching consumer [R8 DataDog/dd-trace-rb PR 6133](DataDog/dd-trace-rb#6133) is also a draft pending this API, the v39 package, and [R3 DataDog/dd-trace-rb PR 6143](DataDog/dd-trace-rb#6143). L4 remains draft until L1 is ready. L0 tracks [APMSP-3830](https://datadoghq.atlassian.net/browse/APMSP-3830); L1-L5 implement the payload work tracked by [APMSP-3219](https://datadoghq.atlassian.net/browse/APMSP-3219). # How to test the change? Prior local validation completed successfully: - `nix develop --command cargo test -p libdd-data-pipeline-ffi --locked` passed 68 tests, including direct-setter/standalone-encoder equivalence, atomic failure, invalid handles, overwrite behaviour, and ownership. - `nix develop --command cargo clippy -p libdd-data-pipeline-ffi --all-targets --locked -- -D warnings` passed. - `nix develop .#nightly --command cargo fmt --all -- --check` passed. - The release artifact built successfully with `nix develop -c cargo run --bin release -- --out /tmp/opencode/libdatadog-m3/x86_64-linux`. - The matching consumer's native suite passed 122 examples, and AppSec passed 807 examples against the local artifact. - `nix develop .#ruby40 --command nix shell nixpkgs#valgrind --command bundle exec ruby_memcheck vendor/bundle/ruby/4.0.0/gems/rspec-core-3.13.6/exe/rspec spec/datadog/tracing/transport/native/tracer_span_spec.rb spec/datadog/tracing/transport/native/send_traces_spec.rb spec/datadog/tracing/transport/native/trace_exporter_spec.rb` passed with no Valgrind memory errors. - Full Standard and Steep checks passed on the matching consumer branch. [APMSP-3219]: https://datadoghq.atlassian.net/browse/APMSP-3219?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ [APMSP-3830]: https://datadoghq.atlassian.net/browse/APMSP-3830?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ Co-authored-by: ekump <edmund.kump@datadoghq.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AI-generated code disclosure: this PR was implemented with substantial AI assistance and manually reviewed and validated.
What does this PR do?
Adds an opaque per-key
meta_structblob setter toTracerSpan. Keys are validated as UTF-8, while values are copied as uninterpreted binary data. This transitional API lets consumers pass their existing per-key MessagePack blobs without changing the Rust trace model.Motivation
Native exporter consumers cannot currently populate
meta_struct, which blocks payload parity tracked by APMSP-3219. The opaque setter provides the safe first stage before the language-neutral structured-value APIs.Additional Notes
Stack position: L1, current PR: DataDog/libdatadog PR 2303.
These open L0-L5 changes are not included in libdatadog v38.0.0. They are planned to ship together in a single libdatadog v39 release, with the common dd-trace-rb uptake tracked by R3 DataDog/dd-trace-rb PR 6143.
Complete expected libdatadog review and merge order for that single v39 release:
L0-L3 are independent and can be reviewed in parallel. Merging them in the listed order is still recommended because it reduces integration and release-branch conflicts, and lands panic containment before the new payload surfaces. L4 is stacked on L1, and L5 is stacked on L4.
The matching payload consumers are R4 DataDog/dd-trace-rb PR 6130, R5 DataDog/dd-trace-rb PR 6129, R6 DataDog/dd-trace-rb PR 6134, R7 DataDog/dd-trace-rb PR 6132, and R8 DataDog/dd-trace-rb PR 6133. All depend through the common v39 uptake in R3 DataDog/dd-trace-rb PR 6143. The downstream review and merge order after R3 is R4, R5, R6, R7, then R8; R7 also depends on R4, and R8 depends on R7.
DataDog/libdatadog-rb v37 PR 65 and DataDog/libdatadog-rb v38 PR 66 are separate ordered prerequisite releases only for R2 DataDog/dd-trace-rb PR 6131 and its
ForkSafeRuntimeuptake. They do not contain the open L0-L5 changes or payload APIs. R3 is stacked on R2, but R3, rather than either prerequisite package, is the common v39 payload-API uptake.L0 tracks APMSP-3830; L1-L5 implement the payload work tracked by APMSP-3219.
How to test the change?
Prior local validation completed successfully:
nix develop -c cargo test -p libdd-data-pipeline-ffipassed, including arbitrary binary values, overwrite semantics, null handles, and invalid UTF-8 keys.nix develop -c cargo clippy -p libdd-data-pipeline-ffi --all-targets -- -D warningspassed.nix develop .#nightly -c cargo fmt --all -- --checkpassed.nix develop -c cargo run --bin release -- --out /tmp/opencode/native-meta-struct-libdatadog/x86_64-linuxand consumed successfully by the matching dd-trace-rb branch.nix develop .#ruby34 --command bundle exec rspec spec/datadog/tracing/transport/native/tracer_span_spec.rb spec/datadog/tracing/transport/native/conformance_spec.rb spec/datadog/tracing/transport/native/transport_spec.rb spec/datadog/appsec/actions_handler/serializable_backtrace_spec.rbpassed 116 examples, including AppSec and AI Guard-shaped wire conformance.