Repository navigation
feat(twilio): a relay session whose playback is stuck is connected again and says what never played - #1500
Merged
yileicn merged 3 commits intoNov 18, 2026
Conversation
…ain and says what never played
On some calls Twilio said the agent started speaking and never that it stopped. Nothing sent after that played, none of
the caller's words came through, and Twilio reported no error, until the caller hung up (2 of 15 calls with a full
closing on an RSP dev line in 40 hours). The session now reads Twilio's speaker events (the events attribute set to
speaker-events) and can recover:
- ConversationRelaySetting.StuckPlaybackSeconds, also per agent: how long past the moment everything said should have
finished playing (estimated from its length, on the slow side) Twilio may still say the agent is speaking, with nothing
cut into and no turn running, before the playback is taken as stuck. 0, the default, turns it off.
- Stuck, the session ends with handoffData {"action":"reconnect","resume":{...}}: the lines said since Twilio last said
the playback stopped (fillers and idle prompts left out), each with whether it went out to play to the end, and the
flow's quiet wait and goodbye on silence. Only while the call may be connected again (MaxReconnects): the session reads
its count from a new "reconnects" Parameter.
- The action callback answers it with a new <ConversationRelay> carrying the resume as a "resume" Parameter, and tells
the hooks nothing, as the call goes on. A call that can't be connected again ends as the agent's.
- The new session says those lines again, each as it went out (they are in the history already, so not kept again),
and takes the quiet wait and goodbye on silence back.
- While it watches, no idle prompt is said while Twilio still plays (it would only queue behind a stuck playback and put
off finding it), the caller's silence starts no sooner than Twilio says the playback ended, and what goes out after
"agentSpeaking off" or a cut-in plays from then, not after what the estimate still had to come.
Tests are in onebrain (UnitTest/TwilioHooks/ConversationRelayWatchdogTests.StuckPlayback.cs), which runs this plugin's
relay session over a fake socket: 71 of 71 relay tests pass, 10 of them new.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Qodo reviews are paused for this user.Troubleshooting steps vary by plan Learn more → On a Teams plan? Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center? |
…g the session Twilio acts on an "end" message only once what is queued has played: on dev, every session we ended with a goodbye and "end" sent together was reported ended 5.4 to 6.0 s later, the goodbye's length (9 calls). A stuck playback never finishes, so the "end" the stuck session sent would never have been acted on and the call would never have been connected again. - The stuck session now keeps what to say again (ConversationRelayResumes: in Redis when there is one, as the callback may run on another instance; on this instance otherwise, for 5 minutes) and drops its socket. Twilio ends the session at once and its callback finds it failed, as when an instance holding a socket goes away, which dev already recovers from (calls a2f3dffa, 469eeb86). - The action callback takes what was kept for the conversation, once, whatever status Twilio gives the dropped session, and connects the call again with it. Not for a session that ended with a handoff (a hang-up or a transfer), nor past MaxReconnects. - The "reconnect" handoff and its handoffData are gone. - A negative StuckPlaybackSeconds takes a playback still going that long before its expected end as stuck, so the reconnect can be tried on a test call: no real stuck playback can be made on demand. Tests (onebrain UnitTest/TwilioHooks/ConversationRelayWatchdogTests.StuckPlayback.cs): the stuck session drops the socket, keeps the unheard lines and sends no "end"; the callback connects the call again with what was kept, once; a hang-up or a call past its reconnects isn't connected again; a negative setting fires while Twilio still plays. 107 of 107 relay tests pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Copilot was unable to review this pull request because the user who requested the review has insufficient remaining quota. Try requesting a review at a lower effort level.
An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.
…io for the speaker events it is found by Review of SciSharp#1500: - The lines to say again are kept apart from the last eight utterances kept for cut-ins, all of them since Twilio last said the playback stopped: a stuck playback plays none of what went out after it, however much that is. - No more falling back to the last line when nothing to say again went out since then: after an idle prompt or a filler that line had already played. The case it was for, a line that crossed Twilio's "agentSpeaking off", is now kept as one: a line that went out less than 1 s before that message is not taken as played, and is waited for before the playback counts as stuck. - What was kept for the new session is read and deleted in one Redis transaction, so two callbacks can't both take it (not GETDEL, which needs Redis 6.2). - A relay watched for a stuck playback asks for Twilio's speaker events itself (events="speaker-events") instead of relying on the host adding the attribute. Tests (onebrain UnitTest.TwilioHooks, built against this branch): every line kept past eight, a line that crossed the "off" is said again and waited for, a stuck idle prompt says nothing again, the events attribute. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Member
|
reviewed |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

Why
On some calls Twilio ConversationRelay said the agent started speaking (
agentSpeaking on) and never said it stopped. After that:clientSpeakingarrived);errormessage and logged no 6410x error;Seen on 2 of 15 calls with a full closing on an RSP dev line in 40 hours. A healthy call with the same lines, the same timing and the caller talking over the first line played normally, so nothing we send tells the two apart.
What changed
All of it is off unless
StuckPlaybackSecondsis set, globally or per agent. When it is set,ReturnConversationRelayasks Twilio for its speaker events (events="speaker-events").Detection
infoframes (agentSpeaking on/off); a cut-in counts as the playback stopping.EstimateSpeech, which runs on the slow side.StuckPlaybackSecondspast that moment, nothing was cut into, and no turn is running.Reconnect
ConversationRelayResumeand drops its socket. The resume holds the lines said since the playback last stopped, each with whether it went out uninterruptible, plus the flow's quiet wait and goodbye on silence.offis kept too: it may have crossed Twilio's message and be the one playing now.reconnects < MaxReconnects. The session reads its count from a newreconnectsParameter on every<ConversationRelay>.end: Twilio acts onendonly once what is queued has played. On dev, every session ended with a goodbye andendsent together was reported ended 5.4–6.0 s later, which is the goodbye's length. A stuck playback never finishes, soendwould never take effect. A dropped socket ends the session at once, and Twilio's callback finds it failed, the same as when an instance holding a socket goes away. Dev already recovers from that case (calls a2f3dffa, 469eeb86).ConversationRelayResumes. It goes to Redis when there is one, because the callback may run on another instance; otherwise it stays on this instance for 5 minutes.TwilioConversationRelayControllertakes the resume for the conversation, once, whatever status Twilio gives the dropped session. In Redis it is read and deleted in one transaction, so two callbacks can't both take it. It answers with a new<ConversationRelay>that carries the resume as aresumeParameter, and tells the hooks nothing, since the call goes on.MaxReconnects.The new session
setupit says those lines again, each with its interruptible flag. They are already in the history, so they aren't kept again.While watching
agentSpeaking off.offor a cut-in, the playback estimate restarts from then, plus the length of any line that may have crossed theoff.Trying it on a call
StuckPlaybackSecondstreats a playback still going that long before its expected end as stuck. No real stuck playback can be made on demand, so this is how to exercise the reconnect on a test call.Limits
StuckPlaybackSecondsafter the estimated end of everything queued, because Twilio reports oneon/offfor lines played back to back. For a three-line closing that is about 35 + 8 s.offis said again, even if the first of them actually played.<Parameter>values. The resume is usually under 700 characters.Tests
The tests are in onebrain, which runs this plugin's relay session and its action callback over a fake socket (
UnitTest/TwilioHooks/ConversationRelayWatchdogTests.StuckPlayback.cs). All 113 relay tests pass:end;offare kept, each with its interruptible flag, and fillers are left out;offare kept, not only the last eight;offis kept, and isn't taken as stuck while it may still play;StuckPlaybackSecondsis set;off;MaxReconnectstimes is not connected again;ReturnConversationRelaycarriesreconnectsandresume.🤖 Generated with Claude Code