Skip to content

Skip no-op payload passes and context lookups without a codec or external storage - #2090

Open
tonypodero wants to merge 1 commit into
temporalio:mainfrom
tonypodero:skip-noop-payload-passes
Open

tonypodero wants to merge 1 commit into
temporalio:mainfrom
tonypodero:skip-noop-payload-passes

Conversation

@tonypodero

Copy link
Copy Markdown

What changed

  • When the data converter has no payload codec and no external storage, workflow activations and completions skip the payload passes that cannot change a payload, and skip resolving a serialization context per command. The check that rejects external storage references without storage configured (TMPRL1105) still runs on every activation.
  • DataConverter._decode_payload_has_effect had no callers. It is now _has_codec_or_external_storage, and both the bridge and the workflow worker use it.

Why

Since 1.23 the payload passes in decode_activation / encode_completion run unconditionally (#1288, #1341). 1.25 split them into two passes per direction (#1394) and 1.35 into three (#1946). From 1.25, each visited payload sequence also asks the workflow instance for its serialization context (#1387). For a sandboxed workflow, each of those lookups is an exec into the sandbox. With the default converter, none of this work changes a payload.

We hit this after upgrading 1.22 → 1.35. Long-running workflows (3k–8k events) are rebuilt from history after each deploy, and those replays became 1.3–1.6× slower, which pushed the largest past the workflow task timeout. On three of our own histories (2.6k–7.4k events), this change brings replay time back to 1.22.0's, 25–38% less than on 1.35.0.

Replay of a synthetic history (activity + 5-minute timer in a loop, default converter, default sandbox), relative to 1.22.0. Median of 3 interleaved rounds × 5 replays; macOS arm64, Python 3.12, protobuf 5.29.6:

3,305 events 7,705 events
1.35.0 1.56× 1.51×
1.35.0 + this change 1.12× 1.15×
Where the time went (1.35.0, 7,705-event history, 1,402 activations)
  • Sandbox _run_code calls per replay: 1,406 on 1.24.0 (one activate per activation), 6,310 on 1.35.0. The extra calls are 4,203 get_serialization_context and 701 get_external_store_context.
  • decode_activation / encode_completion alone, with the default converter, per call: 0.1 / 0.1 µs on 1.22.0 and 10.9 / 12.9 µs on 1.35.0.
  • Replay vs 1.22.0 by version: 1.23 1.03–1.05×, 1.24 1.04–1.07×, 1.25 1.35–1.47×, 1.34 1.39–1.42×, 1.35 1.55–1.58×.
How it was tested
  • New test test_command_context_resolved_only_with_codec_or_external_storage: a workflow without a codec makes no context lookups, and one with a codec still does. It fails on 1.35.0 (3 lookups) and passes with this change.
  • No local Rust toolchain, so I applied the change to the 1.35.0 wheel (these three files match main apart from unrelated lines) and ran the 1.35.0 tests against it:
    • test_serialization_context.py, test_extstore.py and worker/test_extstore.py: 88 passed, the same as unpatched.
    • worker/test_workflow.py, worker/test_replayer.py, worker/test_activity.py, and test_serialization_context.py with the new test: 348 passed, 2 skipped, 0 failed.
  • Not run locally: poe lint-types and the full poe test on main.

@tonypodero
tonypodero requested a review from a team as a code owner November 5, 2026 15:23
@CLAassistant

CLAassistant commented Nov 5, 2026 •

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@tonypodero
tonypodero force-pushed the skip-noop-payload-passes branch from 9bc9d39 to c1a34f7 Compare November 6, 2026 07:25

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants