Skip to content

federation: reuse established tunnel instead of re-dialing peer:9191 - WPB-26951 - #76

Open
bsinou wants to merge 1 commit into
devfrom
WPB-26951_federation-tunnel-reuse-dev
Open

bsinou wants to merge 1 commit into
devfrom
WPB-26951_federation-tunnel-reuse-dev

Conversation

@bsinou

@bsinou bsinou commented Nov 11, 2026

Copy link
Copy Markdown
Collaborator

Port of #59 to dev. The commit is cherry-picked from #59 unchanged and applies cleanly.

When egress NAT rewrites the source port of our federation dial, the peer misses dest:9191 in its send path and dials back into us. That redundant dial clashes with our conntrack entry, the kernel splits the inbound ClientHello across fresh source ports, and the handshake times out: one-way media in federated calls.

The send path now looks up an established tunnel to the peer IP (new peers-by-IP index, registered at handshake completion, unregistered in delete_socket_from_map) and only dials when there is none. One side deploying this heals a pairing.

See #59 for the full analysis, the thread model and the known simultaneous-dial limitation.

Testing

Built on Ubuntu 24.04 as in CI: make check passes, and the TCP, TLS, UDP and DTLS client tests pass. The federation path itself is not covered by these tests; #59 has the staging validation plan.

@bsinou
bsinou requested a review from a team as a code owner November 11, 2026 08:27

@z-dule z-dule left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, and looks good to me.

The federation send path only looked up dest:9191 in children_ss and
dialed a new outbound DTLS connection on a miss. When our egress NAT
rewrites the source port, the peer keys our tunnel under the rewritten
port, so its own send path misses and it dials back into us. That dial
is redundant (a healthy tunnel between the two coturns already exists)
and it is exactly the handshake that fails: our conntrack entry for the
dial occupies the local:9191->peer:9191 tuple, so the kernel's clash
resolution rewrites the inbound ClientHello fragments to a fresh source
port each, and the handshake can never complete (one-way media, paired
10s handshake timeouts on the receiving side).

Maintain a peers-by-ip index (peer IP, port zeroed -> established
federation socket), registered when any federation handshake completes
and unregistered in delete_socket_from_map so entries can never dangle.
On a send-path miss of dest:9191, reuse any established tunnel to that
peer IP instead of dialing; DTLS application data flows both ways
regardless of client/server role (the server-side PONG already proves
this path). Only dial when no established tunnel exists at all.

One side deploying this heals a pairing: the redundant dial never
happens, and the clash precondition is never created on the peer's
node. Same thread model as children_ss (federation listener thread).

(cherry picked from commit 4304282)
@bsinou
bsinou force-pushed the WPB-26951_federation-tunnel-reuse-dev branch from 81853e6 to 610fcb1 Compare November 11, 2026 12:11

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants