Skip to content

fix: keep the track processor across mute and restartTrack - #1249

Draft
hiroshihorie wants to merge 1 commit into
mainfrom
hiroshi/keep-processor-across-restart
Draft

hiroshihorie wants to merge 1 commit into
mainfrom
hiroshi/keep-processor-across-restart

Conversation

@hiroshihorie

Copy link
Copy Markdown
Member

Problem

A TrackProcessor set on a local track is silently dropped in two cases:

  • restartTrack(): it calls stop(), which destroys the processor and clears _processor, before it reads _processor to attach it to the new track. So the processor is always lost. setCameraPosition() and switchCamera() go through restartTrack(), so a camera switch drops a video processor.
  • Mute and unmute with the default options: stopAudioCaptureOnMute and stopCameraCaptureOnMute both default to true. Muting then calls stop(), and unmuting calls restartTrack(), so a microphone or camera processor is gone after the first mute and unmute. Apps that set these options to false only toggle enabled and are not affected.

Fix

mute() and restartTrack() now stop and dispose the media track without releasing the processor, and restartTrack() moves it to the new track as before. stop() still releases the processor, also when a stop-on-mute already stopped the media track. Before, that case skipped stopProcessor().

stop() keeps calling super.stop() itself because Track.stop is @mustCallSuper, so the shared teardown takes its result as a parameter.

Testing

  • New test/track/local_track_processor_test.dart: mute and unmute, restartTrack(), and stop() after a mute. Against main's local.dart, the first two fail with processor null.
  • flutter test test/track/: 21 passed. flutter analyze is clean.
  • Desktop e2e on Linux arm64 in Docker, through a local livekit-server. A screen share has a native video processor attached through a TrackProcessor (flutter_webrtc's desktop AddVideoProcessor, feat(desktop): let native plugins attach video processors to local tracks flutter-webrtc/flutter-webrtc#2233). After restartTrack(), the processor moves to the new MediaStreamTrack and the subscriber keeps decoding processed frames. Without this change, the processor is gone after the restart. Passed 3 runs in a row.

Found while working on CLTCU-14 (desktop video processors for Proton).

restartTrack called stop(), which destroys the processor and clears
_processor, before it read _processor to attach it to the new track, so
the processor was always dropped.

Mute went through stop() too. With the default stopAudioCaptureOnMute
and stopCameraCaptureOnMute (both true), muting stops the media track and
unmuting restarts it, so a microphone or camera processor was lost after
the first mute and unmute, and on every camera switch.

mute and restartTrack now stop the media track and keep the processor,
and restartTrack moves it to the new track as before. stop() still
releases it, also when a mute already stopped the media track.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant