fix: keep delayed duplex transcripts on the original audio burst - #7244
fix: keep delayed duplex transcripts on the original audio burst#7244cpruijsen wants to merge 1 commit into
Conversation
A fragment can arrive after the last audible frame or after the burst has closed; waiting for the next frame dropped it or gave it to the next utterance. A closed burst only claims a span on audio it already forwarded.
|
|
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
There was a problem hiding this comment.
Devin Review found 1 potential issue.
1 flag not posted on this PR by your GitHub settings — view it in Devin Review. (Configure)
| if burst is not self._burst and not ( | ||
| burst.audio_start_ms - _ATTACH_LEAD_MS | ||
| <= fragment.start_ms | ||
| <= burst.audio_end_ms | ||
| ): | ||
| break | ||
| burst.anchor_ms = fragment.start_ms - burst.audio_start_ms | ||
| if fragment.start_ms - burst.anchor_ms > heard_ms + _ATTACH_LEAD_MS: | ||
| break | ||
| elif burst is not self._burst and self._burst is not None: | ||
| break |
There was a problem hiding this comment.
🟡 Delayed transcripts still switch bursts
When a delayed first fragment lacks a timestamp or uses another clock, _attach_due rejects its closed burst. An open next burst consumes that fragment and records the transcript under the wrong message.
Learn more
A transcript span is optional and only guarantees continuity between transcript fragments. It is not guaranteed to share the adapter audio clock, as documented by DuplexOutputTranscriptDelta. A closed burst without an anchor accepts its first delayed fragment only when the raw span falls inside the adapter's audio range. The adapter therefore cannot establish the offset that previously allowed different clocks. A timestamp-less fragment is also explicitly withheld from the closed burst whenever a new burst is open, then immediately accepted by that new burst.
Example: A burst forwards adapter audio from 2,000–2,300 ms, while its provider transcript spans 9,000–9,300 ms. If that first fragment arrives after closure, the range check rejects it. The next audible burst establishes its anchor from 9,000 ms and receives the old words. The same reassignment occurs when start_ms is None and the next burst has opened.
Recommended fix: Preserve burst ownership independently of raw timestamp equality. Use a provider utterance identifier or normalize both streams to one clock when available. For providers without that information, define an ordering or grace policy that keeps a delayed first fragment on the latest eligible closed burst without stealing a confirmed leading fragment for the next burst. Add tests for a closed burst with mismatched clocks and for start_ms=None after the next burst opens.
Was this helpful? React with 👍 or 👎 to provide feedback.
2 participants
Summary
Delayed duplex transcripts attach to the burst that already played that audio, including after the gate has closed. Late text is recorded under the original message id and is not used to open the next burst.
DuplexRealtimeAdapteronly attached queued transcript fragments from_on_audio_framewhile the audio gate was open. A fragment describing audio that had already arrived could miss its utterance:Both orders are in #7227. A third order (next burst already open, then the delayed first fragment) also attached the old text to the new utterance.
The adapter attaches due fragments when they arrive and when the burst closes, same due/ahead rule as before. A closed burst is remembered so late text that still sits on that audio range can be recorded on it.
A closed burst with no offset yet only claims a span in
[audio_start - lead, audio_end]. The extra lead pastaudio_endwould include the next utterance's onset after a normal gated pause. Unplayed spans (audio this burst never reached) are still not published on that item.Decision
Late eligible text is attached to the closed burst's chat item;
text_chis not kept open after audio ends. Keeping the text channel open until a grace timeout or the next burst would leaveforward_generationwaiting on a stream that usually never gets another fragment. Can switch if the stream should stay open instead.A closed burst without an offset may still take a first fragment after the next burst has opened, but only if that span sits on audio it already forwarded (no extra lead past
audio_end). Freezing identity once the next burst opens still gives the next burst the previous utterance. A ±300 ms window around the whole closed range would claim the next utterance after a real silence gap.Test plan
tests/test_duplex_adapter.py: delayed transcript after the last frame still joins that burst (including mismatched span/audio clocks while the burst is still open).when=before_close/in_gap/after_next): original id keeps the text; the next burst does not.Fixes #7227