srt: limit retransmissions to avoid a NAK-driven retransmit storm - #2199
Open
RomanHerbstmann wants to merge 1 commit into
Open
RomanHerbstmann wants to merge 1 commit into
RomanHerbstmann wants to merge 1 commit into
Conversation
Receivers with periodic NAK reports (e.g. gosrt) report every missing packet again on each NAK interval (20 ms minimum). reSendPackets resent all reported packets unconditionally, so under a bottleneck one lost packet could be sent up to latency / 20 ms times and loss amplified itself. - Token bucket for retransmissions: 25 % of the measured media rate by default (libsrt SRTO_OHEADBW), burst of up to 0.5 s of media. - Resend oldest first; stop resending once the budget is exhausted. - Skip packets that cannot arrive before their latency expires and already retransmitted packets reported again within max(rtt + 4 * rttVariance, 20 ms). - setRetransmitOverhead(percent), <= 0 restores the unlimited behavior. - packetsLostUnique counts each lost sequence number once. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
SRT: limit retransmissions
Motivation
Under a network bottleneck the SRT client can multiply its own traffic many times over. Receivers that use periodic NAK reports (e.g. gosrt, used by MediaMTX) report every missing packet again on each NAK interval (20 ms minimum) until the packet is received or its latency expires.
CommandsManager.reSendPacketsresends every packet in each reported range unconditionally, so a single lost packet can be sent up tolatency / 20 mstimes (100 times with a 2 s latency). On a link that is already too slow, the retransmits are lost too and the loss rate climbs.In our measurement (Android emulator → MediaMTX 1.18.2, ingress policed to 250 kbit/s, ~300–450 kbit/s media, latency 2000 ms), about 600,000 packets were offered to the limiter within 150 s, a few percent of them original media packets, and 95 % were dropped.
In addition,
packetsLostincreases on every NAK report, including repeated reports of the same packet, so it overstates the real loss by a large factor and cannot be used as a loss-rate signal.libsrt avoids this by limiting the retransmission overhead relative to the input bandwidth (
SRTO_OHEADBW, default 25 %).Change
CommandsManagerkeeps a token bucket for retransmissions. The rate is a percentage of the measured media rate (default 25 %, minimum 8 kB/s). The bucket allows an immediate burst of up to 0.5 s of media, so short loss events on a healthy link are recovered at once. The average stays at the configured percentage.max(rtt + 4 * rttVariance, 20 ms)(capped at latency / 4). RTT and RTT variance come from the ACK packets.SrtClient.setRetransmitOverhead(percent), exposed asSrtStreamClient.setRetransmitOverhead(percent). A value<= 0restores the previous unlimited behavior.packetsLostUnique/SrtStreamClient.getPacketsLostUnique()counts each lost sequence number once.packetsLostis unchanged.bytesSendstill counts only original packets, so bitrate and stall detection based on it are unaffected.Testing
CommandsManagerTest(time gate, budget and refill, strict oldest-first, packets about to expire, burst on a healthy link, unlimited mode, sequence wrap, unique count, reset).tc police, small burst) on the MediaMTX UDP port for 150 s. Fresh stream per run, same app build except for this change.packetsLost/packetsLostUniqueat the endbytesSendonly counts original packets, so stall detection is unchanged.If you prefer the limit to be opt-in (default
0), that is a one-line change.