Repository navigation
perf(sync): carry every sync source over one socket - #358
Conversation
Chrome opens WebSockets to a host one handshake at a time, and each sync
source had its own socket, so a load opened the user's object, then each
org, in turn: ~6s to channels for a user in 10 orgs (prod trace, 2026-10-31
00:51:30), each upgrade ~60ms on the server and the rest queued in the
browser. The org collections are ready only once every org answered, so the
sidebar waited for the last handshake.
Now the client opens one socket, `GET /sync/mux`, and each source as a
numbered stream on it. The api Worker opens each source's object socket as
before and relays frames unparsed (`@hazel/sync-do` `mux.ts`): effect
`Socket`s for both sides, a `FiberMap` of relays keyed by stream, each
relay's scope owning its object socket. The session starts with the
response (`forkDetach({ startImmediately })`): workerd drops frames that
arrive before a listener does.
On the client, `SyncMux` is each connection's `WebSocket` constructor, so
reconnects, resumes, `Reauth` and close codes are unchanged. The per-source
route stays for older clients.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This comment has been minimized.
This comment has been minimized.
|
Caution Review failedThe pull request is closed. Walkthrough
Merge Risk:
|
Coverage Report
File Coverage
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
…open The relay registered the socket's close only after `open` returned, so a `Close` (or the client going) while the object was still answering interrupted the relay and left the object's socket open. Opening is now the acquire of an `acquireRelease`: uninterruptible, so the interrupt waits for the socket and the release closes it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Merged with 1 step from "Before merge" still open. If it is done, tick it on the review comment; if not, now is the time.
|
Maple review🟢 Confidence 8/10 · likely safe to merge. Held at 8 because earlier reviews of this pull request found 1 defect. Follow-up to the sync mux:
Fixed since the last review
|
Why
From first load to channels took ~6s. A prod trace (2026-10-31 00:51:30,
b048b86) showed the sync sockets opening one after another: the user socket, then 10 org sockets in sorted-id order, each 300–800ms apart. On the server each upgrade took ~60–130ms.Chromium lets only one WebSocket handshake at a time per host IP (RFC 6455 §4.1), and every source had its own socket (TCP + TLS + upgrade each). The org collections become ready only once every org has answered, so the sidebar waited for the last handshake.
What
GET /sync/mux(packages/domain/src/sync/mux.ts): one socket per client. The client opens each source as a numbered stream (Open{id, kind, sourceId, token}/Close{id}). The Worker answersOpened{id}/Closed{id, code, reason}, where the code is the one the source's own socket would have closed with. Data frames are<id>\n<frame>and pass through unparsed. A stream id is never reused, so a late frame from a closed stream can't reach the source's next one.packages/sync-do/src/mux.ts), on the api Worker:Socket.fromWebSocketwraps the client socket and each object socket; a close arrives asSocketCloseErrorwith its code.FiberMapkeyed by stream id holds one relay fiber per stream.Openstarts one,Closeinterrupts it, and the session scope ends them all when the client goes.acquireRelease), so interrupting the relay closes the socket.startMuxSessionforks withforkDetach({ startImmediately: true }). workerd delivers the client's first frames as soon as the 101 returns and drops any that arrive before a listener exists. The workerd test caught this; the fake-socket tests didn't.apps/backend/src/worker/sync-route.ts): token verification, object resolution and the namespace lookup are now one sharedsocketTargetstep for both routes. Each stream is verified with its own token, which carries the expiry the object enforces. A stream whose token belongs to another user is refused with 4003./sync/:kind/:idstays for older clients.libs/effect-db-collection/src/mux.ts):SyncMuxserves as eachSyncConnection'sWebSocketconstructor, so reconnects, resume,Reauth, heartbeats and close codes are unchanged. The shared socket closes after 10s with no streams. Enabled withmultiplex: trueon the app's hub.The objects are untouched; they still hold hibernatable sockets, now opened by the Worker. Cloudflare limits checked: there's no wall-clock cap on a Worker holding a socket, 10k subrequests per invocation by default, and up to 6 handshaking outbound sockets at once (internal handshakes take ~60ms). A DO relay was ruled out because outbound sockets would keep it from hibernating.
Tests
packages/sync-do/test/workerd/mux.test.ts): two objects stream snapshots and live writes over one socket, Ping/Pong goes through, an object's 4001 reaches its stream,Closecloses that object's socket, and closing the client closes them all.packages/sync-do/src/mux.test.ts), client mux tests (libs/effect-db-collection/src/mux.test.ts) and/sync/muxroute tests.Not yet verified
/sync/muxsocket, and compare boot-to-channels time in Maple against the ~6.3s baseline./live/:messageId) still open one socket each and would queue the same way. They could move onto this socket in a follow-up.🤖 Generated with Claude Code
Need help on this PR? Tag
@codesmith-botwith what you need.Summary by CodeRabbit