Skip to content

perf(sync): carry every sync source over one socket - #358

Merged
Makisuo merged 2 commits into
mainfrom
perf/sync-mux
Oct 31, 2026
Merged

Makisuo merged 2 commits into
mainfrom
perf/sync-mux

Conversation

@Makisuo

@Makisuo Makisuo commented Oct 31, 2026 •

Copy link
Copy Markdown
Collaborator

Why

From first load to channels took ~6s. A prod trace (2026-10-31 00:51:30, b048b86) showed the sync sockets opening one after another: the user socket, then 10 org sockets in sorted-id order, each 300–800ms apart. On the server each upgrade took ~60–130ms.

Chromium lets only one WebSocket handshake at a time per host IP (RFC 6455 §4.1), and every source had its own socket (TCP + TLS + upgrade each). The org collections become ready only once every org has answered, so the sidebar waited for the last handshake.

What

  • GET /sync/mux (packages/domain/src/sync/mux.ts): one socket per client. The client opens each source as a numbered stream (Open{id, kind, sourceId, token} / Close{id}). The Worker answers Opened{id} / Closed{id, code, reason}, where the code is the one the source's own socket would have closed with. Data frames are <id>\n<frame> and pass through unparsed. A stream id is never reused, so a late frame from a closed stream can't reach the source's next one.
  • Relay (packages/sync-do/src/mux.ts), on the api Worker:
    • Socket.fromWebSocket wraps the client socket and each object socket; a close arrives as SocketCloseError with its code.
    • A FiberMap keyed by stream id holds one relay fiber per stream. Open starts one, Close interrupts it, and the session scope ends them all when the client goes.
    • Each relay's scope owns its object socket (acquireRelease), so interrupting the relay closes the socket.
    • startMuxSession forks with forkDetach({ startImmediately: true }). workerd delivers the client's first frames as soon as the 101 returns and drops any that arrive before a listener exists. The workerd test caught this; the fake-socket tests didn't.
  • Route (apps/backend/src/worker/sync-route.ts): token verification, object resolution and the namespace lookup are now one shared socketTarget step for both routes. Each stream is verified with its own token, which carries the expiry the object enforces. A stream whose token belongs to another user is refused with 4003. /sync/:kind/:id stays for older clients.
  • Client (libs/effect-db-collection/src/mux.ts): SyncMux serves as each SyncConnection's WebSocket constructor, so reconnects, resume, Reauth, heartbeats and close codes are unchanged. The shared socket closes after 10s with no streams. Enabled with multiplex: true on the app's hub.

The objects are untouched; they still hold hibernatable sockets, now opened by the Worker. Cloudflare limits checked: there's no wall-clock cap on a Worker holding a socket, 10k subrequests per invocation by default, and up to 6 handshaking outbound sockets at once (internal handshakes take ~60ms). A DO relay was ruled out because outbound sockets would keep it from hibernating.

Tests

  • Real workerd (packages/sync-do/test/workerd/mux.test.ts): two objects stream snapshots and live writes over one socket, Ping/Pong goes through, an object's 4001 reaches its stream, Close closes that object's socket, and closing the client closes them all.
  • Relay unit tests (packages/sync-do/src/mux.test.ts), client mux tests (libs/effect-db-collection/src/mux.test.ts) and /sync/mux route tests.
  • Typecheck passes for domain, sync-do, backend, client lib and web. Suites: sync-do 56 + 7 workerd, backend worker 103, client lib 55, web core 49.

Not yet verified

  • No browser run against a live stack. After deploy, check that DevTools shows a single /sync/mux socket, and compare boot-to-channels time in Maple against the ~6.3s baseline.
  • Message live sockets (/live/:messageId) still open one socket each and would queue the same way. They could move onto this socket in a follow-up.

🤖 Generated with Claude Code


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need.

Summary by CodeRabbit

  • New Features
    • Synchronize multiple sources over a single shared connection, with each source maintaining its own stream.
    • Open and close individual source streams without interrupting others; source data and connection status are routed to the appropriate stream.
  • Improvements
    • Reconnecting a source can resume from its last sequence, and authentication or availability issues are reported for the affected stream.
    • The shared connection closes automatically after a period with no active streams.

Chrome opens WebSockets to a host one handshake at a time, and each sync
source had its own socket, so a load opened the user's object, then each
org, in turn: ~6s to channels for a user in 10 orgs (prod trace, 2026-10-31
00:51:30), each upgrade ~60ms on the server and the rest queued in the
browser. The org collections are ready only once every org answered, so the
sidebar waited for the last handshake.

Now the client opens one socket, `GET /sync/mux`, and each source as a
numbered stream on it. The api Worker opens each source's object socket as
before and relays frames unparsed (`@hazel/sync-do` `mux.ts`): effect
`Socket`s for both sides, a `FiberMap` of relays keyed by stream, each
relay's scope owning its object socket. The session starts with the
response (`forkDetach({ startImmediately })`): workerd drops frames that
arrive before a listener does.

On the client, `SyncMux` is each connection's `WebSocket` constructor, so
reconnects, resumes, `Reauth` and close codes are unchanged. The per-source
route stays for older clients.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@maple-review-bot

This comment has been minimized.

@coderabbitai

coderabbitai Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Review in Change Stack →

Caution

Review failed

The pull request is closed.

Walkthrough

This change adds a shared WebSocket protocol for sync sources, implements client and server multiplexing, adds the backend /sync/mux route, and configures the web sync hub to use one socket for multiple sources.

Changes: Multiplexed Sync

Layer / File(s) Summary
Mux protocol and frame contract
packages/domain/src/sync/mux.ts, packages/domain/src/sync/index.ts
Adds schemas for mux control frames, a subprotocol constant, and helpers to encode and decode stream data frames.
Server relay session and workerd integration
packages/sync-do/src/mux.ts, packages/sync-do/src/index.ts, packages/sync-do/src/mux.test.ts, packages/sync-do/test/workerd/worker.ts, packages/sync-do/test/workerd/mux.test.ts
Adds per-source socket relays over a shared connection. The workerd test worker opens Durable Object streams for mux requests. Tests cover frame relay, refusal, stream closure, and cleanup.
Backend mux route and source authentication
apps/backend/src/worker/sync-route.ts, apps/backend/src/worker/sync-route.test.ts
Adds /sync/mux handling and per-source token validation. The existing sync route uses shared target resolution and forwarding logic.
Client mux transport and SyncHub integration
libs/effect-db-collection/src/mux.ts, libs/effect-db-collection/src/hub.ts, libs/effect-db-collection/src/index.ts, libs/effect-db-collection/src/mux.test.ts, libs/effect-db-collection/src/testing/fake-websocket.ts, apps/web-foldkit/src/core/db/sync.ts
Adds a WebSocket-compatible client multiplexer and an optional SyncHub setting for shared sockets. The web client enables multiplexing. Tests cover stream opening, reconnects, token refresh, and idle closure.

Estimated code review effort: 4 (Complex) | ~45 minutes

Change: Feature

Sequence Diagram(s)

sequenceDiagram
  participant SyncHub
  participant SyncMux
  participant SyncRoute as /sync/mux
  participant MuxSession as runMuxSession
  participant DurableObject
  SyncHub->>SyncMux: Create source stream
  SyncMux->>SyncRoute: Open shared WebSocket
  SyncMux->>MuxSession: Send Open control for source
  MuxSession->>DurableObject: Open source socket
  DurableObject-->>MuxSession: Return source frames
  MuxSession-->>SyncMux: Relay frames with stream ID
  SyncMux-->>SyncHub: Deliver frames to source stream
Loading

Possibly related PRs

Merge Risk: Low · up to 07ba2

Cross-user stream requests are currently refused, but that protection lacks a regression test. This is a bounded concern rather than a demonstrated production failure.

Finishing Touches
📝 Generate docstrings
  • Commit to this branch
  • Create a new PR
🧪 Generate unit tests (beta)
  • Commit to this branch
  • Create a new PR
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@maple-review-bot maple-review-bot Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 inline note from Maple's review. The score and summary are in the review comment above.

Comment thread packages/sync-do/src/mux.ts Resolved
@github-actions

github-actions Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Coverage Report

Status Category Percentage Covered / Total
🔵 Lines 69.61% 21016 / 30190
🔵 Statements 69.06% 23841 / 34518
🔵 Functions 59.35% 8312 / 14004
🔵 Branches 56.9% 9877 / 17357
File Coverage
File Stmts Branches Functions Lines Uncovered Lines
Changed Files
apps/backend/src/worker/sync-route.ts 78.14% 73.49% 76.92% 78.78% 125, 175, 185, 251-288, 301, 318-328, 341, 344, 347, 356-357, 361
apps/web-foldkit/src/core/db/sync.ts 29.16% 16.66% 16.66% 31.57% 31-42, 49-53
libs/effect-db-collection/src/hub.ts 90.1% 84.7% 88.05% 91.72% 104-105, 105, 149, 190, 213, 219, 222, 261, 272-276, 298, 333, 349-351, 382-386, 402, 190
libs/effect-db-collection/src/index.ts 100% 100% 100% 100%
libs/effect-db-collection/src/mux.ts 82.97% 60% 88.88% 91.37% 61, 66, 82, 115-118, 126, 135, 146, 157-160, 165, 170, 174, 202-203, 207, 218, 238, 159
libs/effect-db-collection/src/testing/fake-websocket.ts 100% 91.66% 100% 100%
packages/domain/src/sync/index.ts 100% 100% 100% 100%
packages/domain/src/sync/mux.ts 100% 87.5% 66.66% 100%
packages/sync-do/src/index.ts 100% 100% 100% 100%
packages/sync-do/src/mux.ts 95% 72.72% 94.11% 96.36% 113, 127, 153-156
Generated in workflow #1897 for commit 920af6f by the Vitest Coverage Report Action

…open

The relay registered the socket's close only after `open` returned, so a
`Close` (or the client going) while the object was still answering
interrupted the relay and left the object's socket open. Opening is now the
acquire of an `acquireRelease`: uninterruptible, so the interrupt waits for
the socket and the release closes it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Makisuo
Makisuo merged commit e776d31 into main Oct 31, 2026
6 checks passed
@Makisuo
Makisuo deleted the perf/sync-mux branch October 31, 2026 01:31
@maple-review-bot

Copy link
Copy Markdown

Merged with 1 step from "Before merge" still open. If it is done, tick it on the review comment; if not, now is the time.

  • Manual · After deploy, confirm DevTools shows a single /sync/mux socket and boot-to-channels beats the ~6.3s baseline

@maple-review-bot

maple-review-bot Bot commented Oct 31, 2026 •

Copy link
Copy Markdown

Maple review

🟢 Confidence 8/10 · likely safe to merge. Held at 8 because earlier reviews of this pull request found 1 defect.
no findings · tests covered · risk high −1 · 1 defect fixed in earlier rounds

Follow-up to the sync mux: open is now the acquire of an acquireRelease, so a stream closed while its object socket is still being opened closes that socket instead of leaking it. Safe to merge.

  • relay opens the object socket as an acquireRelease acquire, closing it in the release
  • New test covers a Close arriving while the object is still answering

Fixed since the last review

  • ✅ F1 · A stream closed while it is still opening leaks its object socket

920af6f · What was checked, coverage and telemetry details are on the Maple / review check. Reply "won't fix" to dismiss a finding, or mention @maple-review-bot to ask about one.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant