atunnel: close the relay's both ends before returning - #1101
Open
NekoPunch (orangeCatDeveloper) wants to merge 1 commit into
Open
atunnel: close the relay's both ends before returning#1101NekoPunch (orangeCatDeveloper) wants to merge 1 commit into
NekoPunch (orangeCatDeveloper) wants to merge 1 commit into
Conversation
The cancellation path closed upstream and client from a context.AfterFunc goroutine, so a caller that observed the relay return had no guarantee the streams were shut. TestRelayIngressCancellationClosesBothSides raced that goroutine against its own assertions and failed 89 times in 10000 runs under -race.
Keith Mattix II (keithmattix)
approved these changes
Aug 21, 2026
Keith Mattix II (keithmattix)
left a comment
Collaborator
There was a problem hiding this comment.
Thanks for the fit
Aditya Shantanu (aditya-shantanu)
approved these changes
Aug 21, 2026
Aditya Shantanu (aditya-shantanu)
left a comment
Collaborator
There was a problem hiding this comment.
Verified the analysis and the fix.
- The race is exactly as described: the
context.AfterFunccloses ran on a detached goroutine, socancel()returning guaranteed nothing about stream state, and the unbufferedio.Pipewrite in the test raced the relay'sio.Copyagainst the close. Moving the closes into thectx.Done()branch of the relay loop makes "both sides closed before return" an actual postcondition, and the copy goroutines are still unblocked by those same closes (thedonechannel is buffered, so no leak). - Behavior on the normal completion path is unchanged — the relay still doesn't close either side when both copies finish, same as before with
defer stop(). - Ran
go test -race ./internal/atunnel/ -run TestRelayIngressCancellationClosesBothSides -count=5000locally (darwin/arm64): green. Also green with this branch merged into current main,-race -count=50over the relay/serve tests.
LGTM.
Aditya Shantanu (aditya-shantanu)
added a commit
to aditya-shantanu/substrate
that referenced
this pull request
Aug 24, 2026
Adopted from agent-substrate#1101 by @orangeCatDeveloper to unblock CI velocity. Fixes TestRelayIngressCancellationClosesBothSides (agent-substrate#1100): the close ran on a context.AfterFunc goroutine the test never waited for. Co-authored-by: NekoPunch <engineer.jyao@gmail.com>
Collaborator
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1100
TestRelayIngressCancellationClosesBothSidesfails on about 1% of-raceruns, on branches unrelated to the relay.Root cause
The test asserts a close that happens on a goroutine it never waits for.
relayIngressWithHalfCloseclosed both sides from acontext.AfterFunccallback, which runs on its own goroutine.cancel()returning therefore says nothing about whether the streams are closed — and the test asserted right after it.The client side is an unbuffered
io.Pipe, where aWritecannot complete on its own; it needs someone on the other end. Two goroutines can be that someone:AfterFunccallback closes the pipeWritefails → test passesio.Copy, still parked inReadWritereturns nil → test failsThe scheduler picks the winner, so the test is a coin flip.
-racewidens the window by two orders of magnitude — 1 failure per 10000 plain, 89 with-race— and CI runsgo test -race -v ./....The
actor.Readassertion just above races the same close, but carries a one-second deadline and so almost always wins. TheWritehas no such slack, which is why the failure is always line 123.Fix
Production. Close both sides in the
ctx.Done()branch the relay loop already selects on, rather than from a detached goroutine. This turnsinto
which is what a caller can build on, and it drops a goroutine plus its
stop()bookkeeping.Test. Wait for the relay to return, then assert. The
SetReadDeadlineerror check goes with it: once the relay has closed the pipe, setting a deadline on it fails by design, and the deadline was only ever a guard against hanging if the connection had been left open.Nothing changes about the bytes the relay moves — this is a shutdown-ordering fix, not a data-plane one.
Before / after
Linux/x86_64, 4 cores, matching CI. On darwin/arm64 the same runs give 202 failures before and 0 after.