Skip to content

fix(moq-mux): resync TS audio instead of aborting the broadcast - #2751

Merged
kixelated merged 9 commits into
mainfrom
claude/github-issue-2729-6b44bb
Aug 12, 2026
Merged

fix(moq-mux): resync TS audio instead of aborting the broadcast#2751
kixelated merged 9 commits into
mainfrom
claude/github-issue-2729-6b44bb

Conversation

@kixelated

@kixelated kixelated commented Aug 12, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Root cause (moq import ts: one damaged audio frame header aborts the whole broadcast instead of resyncing #2729): the legacy-audio PES loop propagated a header-parse failure straight out of the demuxer (let header = (self.descriptor.parse)(&data[offset..])?). Nothing scanned forward for the next sync word, so a lost frame sync was unrecoverable by construction rather than by policy, and one damaged byte in an MP2/AC-3/E-AC-3 header ended the whole moq import ts session, taking every other track with it. That put legacy audio at odds with the rest of the same demuxer: the TS layer already reacquires packet alignment byte-wise, and the video path resyncs on Annex-B start codes.
  • The trigger doesn't have to be corruption. The tail carried across a PES boundary is spliced onto whatever arrives next, so a dropped PES or a looping file that wraps mid-frame produces the same unparseable join. That is what accumulated 216 publisher restarts on the reporter's feed, one per loop wrap.
  • Fix: scan to the next sync-word candidate and let the following parse confirm it, the way the TS layer confirms a candidate packet against the next one. A resync also re-anchors the timestamp on the current PES: after a splice the carried tail's PTS describes nothing, and at a loop wrap it is stale by the whole file duration.
  • A frame is confirmed before it is trusted. Sync words are short enough to occur by chance in compressed payload (a valid-looking MP2 header lands about every 25 KiB of random bytes, an ADTS one every 5 KiB), so an offset that merely parses is only a guess. A frame is accepted once a second header parses exactly where the frame it declares ends, the same confirm-before-trust rule the TS layer applies to a candidate packet. This covers every offset nothing has vouched for: one reached by a scan, the first frame of a stream (a capture joins mid-stream, so the first PES routed to a PID can open mid-frame), and the first after a seek. Once a frame is confirmed the stream is in sync and the frame after it starts where that one ended, so steady state pays nothing.
  • Bounded, so it can't fail silently. A PID whose payload isn't the codec its PMT declares would otherwise scan forever, publishing nothing while holding a catalog reservation that withholds the catalog for every track. Past a 64 KiB budget the parse error propagates exactly as it always has. Only a confirmed frame resets that budget, which is what makes it hold: an unconfirmed candidate that reset it would let ~2.6 chance MP2 headers per window keep a junk stream scanning forever.
  • AAC had a second, worse bug in the same shape. AacStream never reassembled a frame split across a PES boundary at all: it rejected the PES with "ADTS frame exceeds PES payload". ISO 13818-1 doesn't require access units to align with PES boundaries for AAC any more than for the legacy codecs, so a mux that split one killed the broadcast on well-formed input, with no corruption or discontinuity involved. AAC now carries the tail like the legacy path and gets the same bounded resync.

Advancing the PTS per frame rather than by index * 1024 falls out of the AAC change: after a resync the frames in a PES are no longer a contiguous run to index into. Both loops now share that step and the sync-word scan.

Public API changes

None. codec::legacy is pub(crate), so the new legacy::Descriptor::sync_byte field and the new adts::MIN_HEADER_LEN (pub(super)) are not reachable outside the crate. Resync / Recover / SyncWord are private to container::ts::import.

Test plan

  • just check (exit 0) and RUSTDOCFLAGS=-D warnings cargo doc -p moq-mux clean.
  • cargo nextest run -p moq-mux: 494 passed, 0 failed. Every regression test above was verified to fail with its fix reverted.
  • Ten new tests in container::ts::import:
    • legacy_resyncs_past_damaged_header — the issue's reproducer: a flipped bit in an MP2 sync word is no longer fatal, and the frames either side survive.
    • legacy_resyncs_past_stale_tail_at_a_splice — a mid-frame cut spliced onto unrelated bytes recovers, and the recovered frame re-anchors on the new PES instead of inheriting the stale tail's PTS.
    • legacy_rejects_an_unconfirmed_sync_candidate — a header-shaped sequence planted in a damaged payload is not published. Without confirmation it is emitted as a frame of payload bytes and eats the front of the next real frame, which then disappears.
    • legacy_confirms_the_first_frame_of_a_stream / legacy_confirms_the_first_frame_after_a_seek — joining mid-frame on bytes that parse as a header publishes nothing until the real stream chains.
    • legacy_does_not_bill_a_frame_that_arrives_slowly — a max-size (1728-byte) MP2 frame dribbled in 4-byte PES must not trip the resync budget. Without the refund it fails with "never regained sync after 66066 bytes" despite never being out of sync.
    • legacy_gives_up_when_nothing_ever_parses — the budget still fails a stream that is simply the wrong codec. The junk is seeded with header-shaped bytes, since that is exactly what defeats a budget that any candidate can reset.
    • legacy_survives_a_looping_file_wrap — the production shape replayed against the real ac3.ts capture. Verified it fails without the fix, with the reporter's exact error (missing AC-3 sync word). Also covers a sync word and parser the synthetic tests don't, and asserts every published frame still starts at an AC-3 sync word, so a resync can never land mid-frame.
    • aac_frame_split_across_pes_reassembles — a legally split ADTS frame is reassembled byte-exact and keeps the PTS of the PES it began in.
    • aac_resyncs_past_damaged_header — the AAC counterpart of the first test.

Notes for the reviewer

Two things I deliberately left out, both worth a decision:

  1. A splice still publishes one corrupt frame. Confirmation covers offsets nothing has vouched for. The frame assembled from a stale tail is not one: it sits at a boundary the previous frame vouched for and its header is intact, so it is published as mixed bytes and sync loss is only detected on the frame after it. legacy_resyncs_past_stale_tail_at_a_splice documents the behavior rather than asserting it is correct.
    Relatedly, confirmation catches a false sync whose declared length misses a real boundary. One that happens to land on it is indistinguishable from a real frame, which is why the tests build fragments that do not align.
  2. The terminal case still fails the session, not just the track (question 3 in the issue). Aborting only the affected stream needs a moq_net::Error to abort with, and there is no established mapping for a media-level failure: Transport(String) is documented as the QUIC connection failing, and App(u16) is unused repo-wide. Picking one would be inventing a convention, so I left the bound behaving exactly as today.

No Cross-Package Sync rows apply: this is demuxer robustness inside moq-mux, with no wire, catalog, container-format, or CLI-surface change, so no drafts/ or doc/ update is needed.

Fixes #2729

(Written by Opus 5)

kixelated and others added 3 commits August 11, 2026 16:22
A frame header that fails to parse ended the whole `moq import ts` session:
the `?` in the legacy-audio PES loop propagated out of the demuxer, so one
damaged byte in an MP2, AC-3 or E-AC-3 header took every other track in the
broadcast with it.

Nothing scanned forward for the next sync word, which made a lost sync
unrecoverable by construction rather than by policy, and put legacy audio at
odds with the rest of the same demuxer: the TS layer reacquires packet
alignment byte-wise, and the video path resyncs on Annex-B start codes.

The trigger doesn't have to be corruption. The tail carried across a PES
boundary is spliced onto whatever arrives next, so a dropped PES or a looping
file that wraps mid-frame produces the same unparseable join. That is what
accumulated 216 publisher restarts on the reporter's feed, one per loop wrap.

Scan to the next sync-word candidate and let the following parse confirm it,
the way the TS layer confirms a candidate packet against the next one. A
resync also re-anchors the timestamp on the current PES: after a splice the
carried tail's PTS describes nothing, and at a loop wrap it is stale by the
whole file duration.

Bound the scanning so this can't fail silently. A PID whose payload isn't the
codec its PMT declares would otherwise scan forever, publishing nothing while
holding a catalog reservation that withholds the catalog for every other
track. Past the budget the parse error propagates as it always has.

Fixes #2729

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
AAC never got the frame reassembly the legacy audio path has. ISO 13818-1
doesn't require access units to align with PES boundaries for either, but the
ADTS loop treated a frame running past the end of its PES as an error, so a
mux that split one killed the whole broadcast. That needs no corruption and no
discontinuity: well-formed input was rejected outright.

Carry the partial frame to the next PES, stamped with the PTS from the PES it
began in, and give AAC the same bounded resync as MP2/AC-3/E-AC-3 so a damaged
syncword costs a frame rather than the session.

Advancing the PTS per frame rather than by `index * 1024` falls out of this: a
resync means the frames in a PES are no longer a contiguous run to index into.
The two paths now share that step, along with the scan that finds the next
sync-word candidate.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The synthetic tests build their own elementary streams, so they pin the
mechanism but not the reported failure. This replays it: play part of
`ac3.ts`, wrap to the top, and the audio PES left open at the cut splices onto
the file's first frames.

Without the resync it fails with the exact error off the reporter's feed,
"missing AC-3 sync word". It also covers a sync word and parser the MP2 and
ADTS tests don't, and asserts every published frame still starts at a sync
word, so a resync can never land mid-frame.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: d5ef0215-28f2-4fde-8886-7e9d64c67bf7

📥 Commits

Reviewing files that changed from the base of the PR and between 416c743 and c249ac3.

📒 Files selected for processing (1)
  • rs/moq-mux/src/container/ts/import.rs
🚧 Files skipped from review as they are similar to previous changes (1)
  • rs/moq-mux/src/container/ts/import.rs

Walkthrough

The change adds frame sync bytes to AC-3, E-AC-3, and MP2 descriptors and names the minimum ADTS header length. The transport-stream importer now reassembles AAC and legacy frames across PES boundaries, preserves originating PTS values, advances timestamps from frame durations, and resynchronizes after damaged data with a bounded scan. Seek and finish handling now clear or log partial frames. Tests cover loop wrapping, frame splits, damaged headers, stale tails, false sync candidates, and recovery exhaustion.

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Title check ✅ Passed The title clearly summarizes the main change: TS audio resynchronization replaces broadcast-aborting failures.
Description check ✅ Passed The description directly explains the audio resynchronization changes, recovery limits, timestamp handling, and regression tests.
Linked Issues check ✅ Passed The PR satisfies issue [#2729] by recovering damaged MP2, AC-3, E-AC-3, and AAC streams with confirmed, bounded resynchronization.
Out of Scope Changes check ✅ Passed The changes support [#2729] and add the requested AAC handling without unrelated API, format, wire, or CLI changes.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
✨ Simplify code
  • Create PR with simplified code
  • Commit simplified code in branch claude/github-issue-2729-6b44bb

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (2)
rs/moq-mux/src/container/ts/import.rs (2)

1212-1304: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

The scan logic is bounded and makes forward progress.

Recover::At(found) always returns found >= offset + 1, so the parse loop cannot spin on one offset. keep.max(offset) prevents both a backwards move and the keep - offset underflow. The budget accounting charges only the skipped bytes, not the carried remainder.

One defensive note for a future codec: codec.min_header_len - 1 at Line 1254 panics in debug builds if a descriptor ever declares min_header_len: 0. All current descriptors use 4, 6, or 7, so this is not reachable today. saturating_sub(1) on min_header_len would remove the trap.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@rs/moq-mux/src/container/ts/import.rs` around lines 1212 - 1304, Update
Resync::recover to compute the carried-header length with a saturating
subtraction on codec.min_header_len before calling data.len().saturating_sub,
preventing underflow if a future SyncWord declares a zero minimum header length
while preserving the existing carry and discard behavior.

2663-2688: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Add the matching bounded-recovery test for AAC.

legacy_gives_up_when_nothing_ever_parses covers the legacy path only. The AAC exhaustion path at Lines 1362-1367 builds its error differently: it calls err.context(...) on the anyhow::Error returned by adts::Header::parse, while the legacy path wraps a thiserror value with anyhow::Error::new. That difference is untested, and the AAC scan uses SyncWord::ADTS with sync byte 0xFF rather than a descriptor value.

Feed a PID declared StreamType::AdtsAac with bytes that never yield an ADTS header, then assert the error text contains "never regained sync".

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@rs/moq-mux/src/container/ts/import.rs` around lines 2663 - 2688, Add a
companion bounded-recovery test for the AAC path near
legacy_gives_up_when_nothing_ever_parses, declaring the PID with
StreamType::AdtsAac and feeding repeated PES payloads containing bytes that
never produce an ADTS header or 0xFF sync word. Drive decoding until it returns
an error, then assert the error text contains "never regained sync", covering
the adts::Header::parse context path.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@rs/moq-mux/src/container/ts/import.rs`:
- Around line 1212-1304: Update Resync::recover to compute the carried-header
length with a saturating subtraction on codec.min_header_len before calling
data.len().saturating_sub, preventing underflow if a future SyncWord declares a
zero minimum header length while preserving the existing carry and discard
behavior.
- Around line 2663-2688: Add a companion bounded-recovery test for the AAC path
near legacy_gives_up_when_nothing_ever_parses, declaring the PID with
StreamType::AdtsAac and feeding repeated PES payloads containing bytes that
never produce an ADTS header or 0xFF sync word. Drive decoding until it returns
an error, then assert the error text contains "never regained sync", covering
the adts::Header::parse context path.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 326faa42-09e5-4312-9231-4488bf2b9bc2

📥 Commits

Reviewing files that changed from the base of the PR and between 05c7bf3 and 25890a0.

📒 Files selected for processing (6)
  • rs/moq-mux/src/codec/ac3.rs
  • rs/moq-mux/src/codec/eac3.rs
  • rs/moq-mux/src/codec/legacy.rs
  • rs/moq-mux/src/codec/mp2.rs
  • rs/moq-mux/src/container/ts/adts.rs
  • rs/moq-mux/src/container/ts/import.rs

kixelated and others added 2 commits August 11, 2026 20:47
Accepting the first offset whose header parses is not safe. A sync word is
short enough to occur by chance in compressed payload: valid-looking MP2
headers land about every 25 KiB of random bytes and ADTS ones every 5 KiB, so
scanning across a damaged region reliably finds false ones.

Two consequences, both silent. The false frame is published as audio and eats
the front of the real frame behind it, which then never appears. Worse, each
false positive counted as a recovery and reset the discard budget, so the
guard meant to stop a misdeclared PID from scanning forever never fired: at
~2.6 false headers per 64 KiB window an MP2 scan resets faster than it
accumulates.

Require a second header exactly where the candidate's frame ends before
publishing it or resetting the budget, which is the same confirm-before-trust
rule the TS layer applies to a candidate packet. Only frames reached by a scan
pay for it: once one is confirmed the stream is back in sync and the frame
after it starts where that one ended. When the buffer is too short to confirm,
the candidate is carried to the next PES instead of being trusted.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The bounded-recovery test only covered the legacy codecs. AAC reaches the same
budget by a different route: it scans for the ADTS sync byte rather than a
descriptor's, and builds its error by adding context to the `anyhow` one
`adts::Header::parse` returns, where the legacy path wraps a `thiserror` value.

Like its legacy counterpart the junk is seeded with header-shaped bytes, so the
test fails if an unconfirmed candidate is ever allowed to reset the budget.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@kixelated

Copy link
Copy Markdown
Collaborator Author

Thanks. Both nitpicks were in the review body rather than inline threads, so replying here.

AAC bounded-recovery test — done (e007df8). Good catch: that path really was untested, and it differs from the legacy one in exactly the two ways you noted (SyncWord::ADTS rather than a descriptor's sync byte, and err.context(...) on an anyhow::Error rather than wrapping a thiserror value). Added aac_gives_up_when_nothing_ever_parses. Like its legacy counterpart the junk is seeded with header-shaped bytes every 16 bytes, whose declared 47-byte frames never end on another header, so the test also fails if an unconfirmed candidate is ever allowed to reset the budget. Verified it fails when confirmation is disabled.

min_header_len - 1 underflow — not changing it, and I'd rather leave the trap than paper over it. A min_header_len of 0 doesn't just underflow here; it makes the parse loop condition offset + 0 <= data.len() permanently true, so a zero-length descriptor spins forever regardless. saturating_sub would convert a loud debug panic at the point of the bad config into a silent infinite loop somewhere less obvious. The three descriptors are statics in this crate at 4, 6, and 7, so the value can't come from outside the tree.

🤖 Addressed by Claude Code

(written by Opus 5)

kixelated and others added 3 commits August 12, 2026 08:42
…a false length

Two gaps in the confirm-before-publish rule, both found reviewing it.

A stream started out trusting its first frame, on the reasoning that a PES
payload begins at a frame boundary. It doesn't have to: ISO 13818-1 lets a
frame span the boundary, so a capture joining mid-stream can be handed the
back half of one. Nothing vouches for that offset, which is the same position
a scan is in, and a chance header there is worse than a corrupt frame: the
track takes its sample rate and channel count from it for the whole broadcast.
Start unconfirmed, and treat a seek the same way, since a discontinuity
retires whatever vouched for the next boundary.

That leaves nothing to confirm the last frame before end of stream, so finish
drains the carried candidate: nothing more can arrive to vouch for it, and a
whole frame that parses beats dropping it.

The second gap: an unconfirmed candidate was parked until its declared length
arrived. A false sync can claim a length the stream never delivers, and byte 5
of a real ADTS header is 0xFF often enough to land one that claims 2184 bytes,
so the parser sat waiting while real frames inside that range went by. Prefer
any later candidate that fits and confirms, and keep the over-long one only as
a fallback if nothing else in the buffer does.

Confirmation catches a false sync whose declared length misses a real
boundary. One that happens to land on it is indistinguishable from a frame,
which is why the tests build fragments that do not align.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
All three found reviewing the previous commit, and all three are ways the rule
leaked rather than ways it was wrong.

The end-of-stream drain accepted an unconfirmed candidate unconditionally,
which handed back exactly what starting unconfirmed rejects: a capture that
joins mid-frame and ends before another frame arrives published the false
frame and built the track's config from it. Drain only once the stream has
published something, so the config always comes from a confirmed frame.

The resync budget billed bytes it went on to keep. A candidate too long for
the buffer is rescanned every time the buffer grows, and each pass charged the
whole buffer, so one 1728-byte MP2 frame arriving in small PES ran up 66 KiB
of "discarded" against a 64 KiB budget and failed a stream that was never out
of sync. Refund the scan when its bytes are retained.

The same path re-anchored the timestamp on the PES it happened to be rescanned
in. A frame spanning three or more PES would take the second one's PTS instead
of the one it began in. Re-anchor only where the tail is genuinely abandoned.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
A rendition is reserved when the PMT announces it and consumed when its
importer is built, and a live reservation withholds the initial catalog
publish for the whole broadcast. `finish` takes streams by reference and
callers keep the importer afterwards (moq-srt finishes it, then ends the
broadcast), so a PID whose importer was never built kept the catalog shut for
every other track until the importer itself was dropped.

Confirming the first frame made that reachable with well-formed input: a PID
carrying a single frame nothing can confirm now publishes nothing, which is
intended, but it must not take the catalog down with it. Release the
reservation at finish when no importer was built.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@kixelated
kixelated enabled auto-merge (squash) August 12, 2026 17:46

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@rs/moq-mux/src/container/ts/import.rs`:
- Around line 1609-1617: Update Resync::desynced to clear discarded along with
marking the stream desynchronized, resetting the recovery scan budget for both
seek paths. At rs/moq-mux/src/container/ts/import.rs:1609-1617, rely on this
shared reset in AacStream::seek; at
rs/moq-mux/src/container/ts/import.rs:1981-1989, make no direct change and
confirm LegacyStream::seek uses desynced rather than clearing the budget
separately. Add a regression test that consumes most of the budget, seeks, then
imports a valid stream that publishes frames.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 5f13ecd3-436d-4e1b-991f-fba1b672ea91

📥 Commits

Reviewing files that changed from the base of the PR and between e007df8 and 416c743.

📒 Files selected for processing (1)
  • rs/moq-mux/src/container/ts/import.rs

Comment thread rs/moq-mux/src/container/ts/import.rs
@kixelated
kixelated disabled auto-merge August 12, 2026 17:51
The budget measures how long the current run of a stream has gone without
finding a frame. A seek ends that run, but `desynced` only marked the next
boundary unconfirmed and left the count alone, and nothing but publishing a
frame clears it.

So a stream that scanned most of the way through the budget, then seeked away
from the damage, died on its first parse failure after the seek with "never
regained sync" while being perfectly in sync. Landing mid-frame is enough to
produce that failure, which is the normal case for a seek.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

moq import ts: one damaged audio frame header aborts the whole broadcast instead of resyncing

1 participant