Skip to content

[Bug]: OTel operation spans report DurableOperationException instead of the original error type #847

Closed

Description

@zhongkechen

Expected Behavior

In both Execution and Invocation trace views, a failed durable operation's exception event should report the original stored error type in exception.type when ErrorObject.errorType is available.

Scenario Affected operation span Expected exception.type
Child-context handler throws RuntimeException otel-failed-child-context java.lang.RuntimeException
Condition function throws RuntimeException otel-failed-condition java.lang.RuntimeException
Callback fails with error type RejectedError otel-failed-callback and otel-failed-callback-callback RejectedError
Invoked target throws RuntimeException otel-failed-invoke java.lang.RuntimeException

Java should retain its existing exception reporting on Workflow and Invocation spans. The exceptions observed at those boundaries can legitimately be CallbackFailedException or InvokeFailedException. Adding equivalent Workflow/Invocation exception reporting to JS and Python is planned as a separate follow-up.

Actual Behavior

The affected operation spans report software.amazon.lambda.durable.exception.DurableOperationException, even though the original error type is present in the stored ErrorObject. For a failed condition, the attempt span already reports java.lang.RuntimeException, while the enclosing operation reports the wrapper type.

This is a telemetry attribution problem; the requested fix should preserve SDK exception objects and checkpoint contents.

Steps to Reproduce

  1. Configure either InvocationOtelPlugin or ExecutionOtelPlugin with an OpenTelemetry SdkTracerProvider and InMemorySpanExporter.

  2. Run a handler with LocalDurableTestRunner that throws from a child context:

    var runner = LocalDurableTestRunner.create(
            String.class,
            (input, context) -> context.runInChildContext("otel-failed-child-context", String.class, child -> {
                throw new RuntimeException("Intentional child-context failure");
            }),
            durableConfigWithOtelPlugin);
    runner.run("input");
  3. Inspect the exception.type attribute of the exported otel-failed-child-context span's exception event.

  4. Observe software.amazon.lambda.durable.exception.DurableOperationException instead of java.lang.RuntimeException.

  5. Repeat with a failing waitForCondition, a waitForCallback completed using failCallback with ErrorObject.errorType("RejectedError"), and an invoke completed using failChainedInvoke with ErrorObject.errorType("java.lang.RuntimeException").

SDK Version

Reproduced locally against Java SDK 3.0.1-SNAPSHOT, main commit 64339d4f, with OpenTelemetry 1.67.0.

Java Version

Is this a regression?

Unknown; no last working version has been established.

Additional Context

The code path explains the mismatch:

  • BaseDurableOperation.extractErrorFromOperation() represents stored operation errors with DurableOperationException, preserving the original ErrorObject.
  • OtelPluginSupport.applyOperationStatus() records the wrapper directly. Its timestamped exception-event path also derives exception.type from the wrapper's Java class.
  • The original type remains available through DurableOperationException.getErrorObject().errorType().

The behavior was reproduced locally through the real SDK execution engine and OpenTelemetry provider. Callback and invoke completion were supplied by the local test backend. The reported live AWS/X-Ray traces have not been independently inspected, and no updated deployment has been validated.

Acceptance criteria:

  • Operation exception events use the stored original error type when supplied, including remote-language types such as RejectedError, in both trace views.
  • Preserve the existing fallback when no stored error type is available.
  • Preserve exception-event counts, attempt reporting, message/stack details, event timestamps, and Workflow/Invocation exception reporting.
  • Preserve caller-visible failures, serialized checkpoints, and replay behavior; completed user code must remain skipped on replay.

Related closed prototype: #846. This issue retains only the operation-type correction from that prototype; Java's existing Workflow/Invocation exception reporting remains part of the intended behavior. JS/Python parity is a later, separate change.

Activity

  1. added
    pkg:otelModule: otel-plugin
    and removed on Oct 30, 2026
  2. zhongkechen commented on Nov 2, 2026

    @zhongkechen
    ContributorAuthor

    Severity - low

    When it triggers. A failed child context, condition, callback, or chained invoke supplies a stored ErrorObject with an original error type. In both OTel views, the operation exception event reports DurableOperationException instead of that stored type.

    Impact. Exception-type searches and grouping combine distinct failures under the SDK wrapper class, making diagnosis harder. A condition attempt can report the original type while its enclosing operation reports the wrapper, so the two levels disagree.

    What is correct. The operation is still marked as an error and still carries an exception event. The original ErrorObject remains available in the SDK; the defect is in the telemetry projection, not the stored failure or caller-visible exception. Workflow and Invocation exception reporting is a separate boundary and should retain its existing behavior.

    Why this severity. This is a diagnostic attribute defect, with the failure and affected operation still visible. It is narrower than missing completion spans or an incorrect terminal status, so low fits better than medium. It still merits correction because the SDK already has the original type, including remote-language error names, and consumers cannot recover it reliably by grouping on the exported wrapper type.

    Assessment is based on the issue's local reproduction and the exception-recording path at 4b1aa82b; no new cloud reproduction was run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingpkg:otelModule: otel-plugin

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions