Skip to content

STOR-5489: Implement retry loop for actor fetch requests - #7064

Open
apeacock1991 wants to merge 5 commits into
mainfrom
apeacock/stor-5489-replay
Open

STOR-5489: Implement retry loop for actor fetch requests#7064
apeacock1991 wants to merge 5 commits into
mainfrom
apeacock/stor-5489-replay

Conversation

@apeacock1991

@apeacock1991 apeacock1991 commented Aug 19, 2026

Copy link
Copy Markdown
Contributor

When the enforcement gate is enabled, we now attempt to retry disconnected actor requests within the following bounds:

  • The request is eligible for retries (e.g. is fetch, is rewindable, is WebSocket upgrade request)
  • Retry up to maximum of 5 times
  • Retry up to a maximum of 10 seconds, with backoff

There's some cleanup in this PR too:

  • Renamed supportsActorRetryMetadata to supportsActorFetchRetries, ultimately this is what we care about vs metadata
  • Extracted out some existing tests to a new test suite that is specifically for retries

Descoped from this PR:

  • Replicas, which I'll tackle separately (there is some complexity around their channel)
  • Adjusting how subrequests counts/metrics are handled (we won't count retries in the request-based limit)

@apeacock1991
apeacock1991 requested review from a team as code owners August 19, 2026 12:13
@apeacock1991
apeacock1991 force-pushed the apeacock/stor-5489-replay branch 2 times, most recently from 05599f4 to 14373e2 Compare August 19, 2026 13:55
@codspeed-hq

codspeed-hq Bot commented Aug 19, 2026

Copy link
Copy Markdown

Merging this PR will degrade performance by 9.24%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

❌ 1 regressed benchmark
✅ 71 untouched benchmarks
⏩ 129 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Benchmark BASE HEAD Efficiency
simpleStringBody[Response] 22.4 µs 24.7 µs -9.24%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing apeacock/stor-5489-replay (0cec5a1) with main (6dd2348)

Open in CodSpeed

Footnotes

  1. 129 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@apeacock1991
apeacock1991 force-pushed the apeacock/stor-5489-replay branch 2 times, most recently from bd4d554 to 38a5707 Compare August 19, 2026 15:00
@apeacock1991 apeacock1991 changed the title STOR-5489: Implement retry loop for actor fetch requests [WIP] STOR-5489: Implement retry loop for actor fetch requests Aug 19, 2026
@apeacock1991
apeacock1991 force-pushed the apeacock/stor-5489-replay branch 4 times, most recently from 4b9ecd6 to 0cec5a1 Compare August 20, 2026 14:22
Move the existing actor retry metadata and factory tests into their
own target. Leave body rewindability coverage isolated.
Use supportsActorFetchRetries() for the existing metadata capability so the name also covers replay behavior added separately.
Add a retry-preparation hook to actor fetch factories. Global actor stubs drop and recreate their channel while preserving routing inputs; replica stubs retain their pre-opened primary channel.
@apeacock1991
apeacock1991 force-pushed the apeacock/stor-5489-replay branch from 0cec5a1 to e644ddb Compare August 21, 2026 10:47
@apeacock1991 apeacock1991 changed the title [WIP] STOR-5489: Implement retry loop for actor fetch requests STOR-5489: Implement retry loop for actor fetch requests Aug 21, 2026
Retry eligible Durable Object HTTP fetches with bounded jittered
backoff while preserving logical-call metadata, rewinding request
bodies, and honoring delivery, abort, and deadline semantics.

Keep replica-to-primary stubs excluded because their pre-resolved
channels cannot reroute after a disconnect.
Apply Durable Object fetch replay to disconnected WebSocket upgrade
handshakes while retaining the existing successful upgrade and HTTP
fallback behavior.
@apeacock1991
apeacock1991 force-pushed the apeacock/stor-5489-replay branch from e644ddb to 3dbfcea Compare August 21, 2026 12:46
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant