AEC in Unity Audio - #381
Merged
Merged
Conversation
MaxHeimbrock
changed the title
libwebrtc instantiates the high-pass filter whenever AEC or NS is enabled (echo_canceller.enforce_high_pass_filtering defaults to true) and the platform ADM path always runs it, so the flag only ever mattered with AEC and NS both off. Hardcode it on for the Unity-audio path so both paths match, and keep the FFI wrapper off the public surface: sources configure it through AudioProcessingOptions. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The struct and RtcAudioSource.AudioProcessingStats only mirrored counters the processor kept for diagnostics. Nothing in the processing path read them, and the only consumers were Meet's periodic log and two test assertions. The one signal that would otherwise go silent, the module rejecting a chunk, now warns once from the main thread the same way the unsupported-rate bypass does. Tests: the chunk count moves to the AudioProcessor level through a counting sink; the source-level test keeps the push path covered. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Remote AudioSources play at Unity's default volume again. The 0.7 came over from the review package as their tuning; on-device testing at full gain was acceptable, and the sample should start with fewer knobs. The custom editor loses its Unity Audio section and shows the AEC3 note directly under the processing toggles when PlatformAudio is off. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MaxHeimbrock
commented
Sep 17, 2026
MaxHeimbrock
commented
Sep 17, 2026
MaxHeimbrock
commented
Sep 17, 2026
MaxHeimbrock
marked this pull request as ready for review
September 17, 2026 13:07
MaxHeimbrock
requested review from
cloudwebrtc,
ladvoc and
xianshijing-lk
as code owners
September 17, 2026 13:07
- Meet scene: point the token config back at the tracked asset instead of a gitignored local config - Meet sample: restore the Publish Data button and drop a duplicate log; stop calling PlatformAudio the default in the class comment - RtcAudioSource: keep feeding the processing stage while muted so AEC3 sees the near end next to its reference; the processed output is dropped instead - AudioProcessor: only run the maintenance loop for echo cancellation and only when a coroutine host exists, since the loop never finishes and would otherwise be drained synchronously; use unscaled time; check the sample rate per block instead of latching a bypass; Dispose(bool) keeps the finalizer path off PlayoutReference state - PlayoutReference: fix the doc header, add DisallowMultipleComponent Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
xianshijing-lk
left a comment
xianshijing-lk
left a comment
Contributor
There was a problem hiding this comment.
The review is still ongoing, I am posting what I have now.
| /// <summary>Main thread. The component stays on the listener; it is inert without consumers.</summary> | ||
| internal static void Release() | ||
| { | ||
| if (_consumers > 0) _consumers--; |
Contributor
There was a problem hiding this comment.
what will happen if _consumers == 0 ?
Contributor
Author
There was a problem hiding this comment.
I had another look and realized this was a bit confusing. I now coupled the Acquire with subscribing to the event delegate AudioRead. I also renamed the _listener to _sceneAudioListener because it was easily confused for the _consumer with that name.
Now the delivery is also clearly coupled to having a _sceneAudioListener as well as consumer, which we don't count ourselves but just look if we have event subscriber. Beforehand it was already doing that in AudioRead?.Invoke because of the ?, but the _deliver flag was kind of split that way.
I hope this has a better readability.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
Unity Audio had no echo cancellation, only
PlatformAudiohad AEC.There is a new
PlayoutReferencewhich captures the final game audio from theAudioListenerin the scene. That is the far-end input for the AECWith audio processing enabled, the
RtcAudioSource(which represents our microphone) creates anAudioProcessor, which itself holds theAudioProcessingModule(APM) wrapping the native APM from WebRTC through FFI. We already had all the APM stuff exposed through FFI anyways, so this is not in this PR.The
AudioProcessorreceives the mic input fromRtcAudioSourceand the scene audio from thePlayoutReference plus an occasional new delay hint (every 2s) and does its processing using itsAudioProcessingModule`, before sending the audio input to the native side.With audio processing disabled, the input audio still goes straight to the native side.
API surface
AudioProcessingOptionsbool PreferHardwarePlatformAudioSourceonly.MicrophoneSourceMicrophoneSource(string, GameObject, AudioProcessingOptions)RtcAudioSourceprotected RtcAudioSource(RtcAudioSourceType, AudioProcessingOptions)RtcAudioSourceprotected RtcAudioSource(RtcAudioSourceType, uint, uint, AudioProcessingOptions?)RtcAudioSourcebool AudioProcessingEnabled { get; }RtcAudioSourceprotected void ResetAudioProcessing()PlayoutReferenceMonoBehaviourAudioListenerwhen echo cancellation is on.Nothing removed or changed in signature.
AudioProcessingOptionskeeps its three stage flags; The FFI wrapperAudioProcessingModuleisinternal; the public entry point isMicrophoneSource/RtcAudioSourcewithAudioProcessingOptions.Verification
Tested manually on MacOS, iOS and Android