feat(ios): real echo cancellation/NR via native Voice-Processing engine
iOS "voice chat" had echo and no noise suppression: real iOS AEC/NS/AGC come only from Apple's Voice-Processing I/O unit (VPIO), but the core plays/captures via miniaudio's plain RemoteIO units, so .voiceChat mode alone never engaged AEC. Core (ABI PATCH 1->2): - vc_set_mixed_output_sink + vc_set_external_playback. In external mode the AudioEngine opens no hardware playback device; a mixer-timer thread drives on_playback (decode+mix) on a ~20ms cadence and ships the final mix to the sink. start() also skips the hardware capture device when the MIC stream is external_feed (AudioParams.external_capture). - New white-box test test_external_playback (drives the timer with no hw). iOS/Swift: - StreamDescriptor.externalFeed; VoiceCatClient.setMixedOutputSink / setExternalPlayback wrappers. - IOSVoiceProcessingEngine: AVAudioEngine + setVoiceProcessingEnabled; mic tap -> feedPcm, mixed-sink lock-free ring -> AVAudioSourceNode (both share the VPIO unit so AEC has its reference signal). - IOSAudioRouter.currentConfigUsesVoiceProcessing scopes VPIO to the AEC presets; SessionState join/leave + reconcileVoicePath() switch paths; Voice Chat defaults to speaker; Settings surfaces AEC/NS state. Known: pending on-device verification; a few bugs to fix afterward.
This commit is contained in:
@@ -215,6 +215,23 @@ Each receiver keeps an **adaptive jitter buffer per ssrc**.
|
||||
`AVAudioSession`, then restarts the devices (`vc_audio_restart`) so they reopen against the
|
||||
new route — mirroring TeamTalk5's `closeSoundDevices`/`initSoundInputDevice`/
|
||||
`initSoundOutputDevice` pattern.
|
||||
- **iOS voice processing (AEC/NS/AGC) — native VPIO path.** Real iOS echo cancellation, noise
|
||||
suppression and AGC are provided ONLY by Apple's **Voice-Processing I/O audio unit (VPIO)**,
|
||||
*not* by the `AVAudioSession` mode alone. The core uses miniaudio's plain `RemoteIO` audio
|
||||
units, which never engage VPIO — so `.voiceChat` mode by itself yields no AEC. For VPIO to
|
||||
cancel echo it must own BOTH the mic capture and the remote-audio playback (it subtracts the
|
||||
played-back signal from the mic), so on the AEC presets (Voice Chat / Bluetooth Headset HFP /
|
||||
Wired Headset) the Swift layer runs a native `AVAudioEngine` with
|
||||
`inputNode.setVoiceProcessingEnabled(true)` and the core runs in **external mode**:
|
||||
- **Mic:** the MIC stream is started with `vc_stream_desc.external_feed=1`; the VPIO input tap
|
||||
feeds processed mic PCM via `vc_stream_feed_pcm`. The core skips its hardware capture device
|
||||
(`AudioParams.external_capture`).
|
||||
- **Playback:** `vc_set_external_playback(1)` makes the core skip its hardware playback device;
|
||||
a mixer-timer thread drives decode+mix on a ~20 ms cadence and delivers the FINAL mixed PCM
|
||||
via `vc_set_mixed_output_sink`. The Swift engine renders that through the VPIO output, so
|
||||
VPIO has its echo-cancellation reference signal.
|
||||
The Stereo Mic / Studio / A2DP presets keep the miniaudio path (they want raw / stereo /
|
||||
no-AEC routing that VPIO can't provide — VPIO forces mono).
|
||||
- **DSP engine: see §11.** The original plan was `webrtc-audio-processing` (AEC + NS + AGC +
|
||||
VAD in one tuned module, BSD-licensed) — but it has no working Windows/MSVC build upstream
|
||||
(confirmed via its own issue tracker: GCC-only Meson build, MinGW support unfinished, hard
|
||||
|
||||
Reference in New Issue
Block a user