Preserve entity unlocks on payload failures and skip known tokens for .NET parity #307

Please bring orchestration payload-failure cleanup and known-token handling into parity with the fixes proposed in durabletask-dotnet PR 816. Source inspection of current main (0262c53c8f13401bfe2c25a3e058128963ad51a9) identifies both gaps below. This is a source-backed report and a regression recipe, not a claim of a live Azure reproduction.

Findings

Entity releases are discarded by the failed-completion fallback.

• externalizeOrchestratorResponse constructs a new response containing only CompleteOrchestration(Failed) when externalization throws. None of the existing actions are retained.
• Those actions can include real lock releases: TaskOrchestrationExecutor emits either native SendEntityMessage.EntityUnlockSent or legacy entity-addressed SendEvent(name="release").
• The rewrite occurs in the outbound interceptor, after the executor produced its actions. If a pending entity call or custom status fails payload externalization while the orchestration still holds locks, changing the wire response to terminal also requires generating the outstanding releases; preserving only releases already present is not sufficient.

Already-externalized values can be uploaded again.

• maybeExternalize checks byte size and uploads without checking payloadStore.isKnownPayloadToken.
• A recognized token at or above the threshold is therefore treated as ordinary data when it reaches this helper again. It can become a nested token, or fail the configured maximum-size check even though no upload is necessary.
• Java protobuf requests are immutable, so this report does not assume every ordinary transport retry mutates/reuses the externalized request in the same way as .NET. The missing invariant is that externalizing an already-tokenized value must be idempotent.

Regression recipes

1. Extend the existing interceptor failure test with two native unlock actions, then separately with two legacy entity releases. Force a permanent PayloadStorageException during custom-status or completion-result upload. The current fallback contains one Failed action; it should retain the two releases before that action and discard unrelated unpersistable work.
2. Replay an orchestration that acquired two locks and is waiting on an oversized entity call. Force a permanent payload failure after replay. The failed completion must also contain releases for the still-held locks, even when no releases existed in the original response.
3. Make a fake store recognize a token whose UTF-8 length exceeds the threshold. Re-externalize a response/request containing that token. The token must remain unchanged and upload must not be called, including when the token itself exceeds the ordinary raw-payload cap.

Acceptance Criteria

• Permanent orchestration payload failures preserve native and legacy entity releases, including their destinations and critical-section/owner identity, before a single terminal Failed action.
• Terminal failures introduced after orchestration execution generate any remaining acquired-lock releases before sending; they do not merely preserve releases already emitted.
• Known payload tokens bypass upload and raw-payload size limits across the supported outbound payload fields.
• Regression tests cover zero/partial/all previously emitted releases, terminal output and pending entity-call failures, and a second caller progressing after the releases are processed.
• Transient storage failures and cancellation are not converted into permanent terminal failures, and completion retries do not duplicate releases.

Related

• .NET parity reference: PR 816, currently proposed rather than a released fix.
• Java PR 306 addresses transient upload classification and other historical parity differences. Its inspected interceptor patch still builds a single-action response for permanent orchestration failures and does not add the known-token upload guard, so this report tracks a distinct remaining gap.