fix(ios-audio): unify iOS audio onto one always-external AVAudioEngine

The iOS audio path was a hybrid: Voice-Chat-class presets ran a native
VPIO AVAudioEngine (core external) while Stereo/Studio/A2DP presets ran
the core's miniaudio devices. Nearly every "no input / no output / both"
bug lived in the seam between the two paths — the lingering miniaudio
capture unit fighting VPIO, the audioRestart ordering dance, the
route-change "glitching" loop, stereo<->mono stickiness, and
"can't hear anyone". Switching presets/routes mid-call routinely dropped
a direction.

Drive ALL iOS audio through one AVAudioEngine with the core fully
external at all times: setExternalPlayback(1) once at connect, every MIC
stream external_feed=1, mic via vc_stream_feed_pcm, playback via
vc_set_mixed_output_sink (drained by an always-on AVAudioSourceNode so
remote audio plays before joining voice). VPIO + AGC toggle per preset.
Every preset/route/interruption change funnels through one deterministic
Swift-only reconfigure (stop -> apply session config -> rebuild -> start)
— no second path to hand off to, so a change can't drop a direction.

- IOSVoiceProcessingEngine.swift -> IOSAudioEngine: always-on source-node
  playback, conditional mic tap, VPIO/AGC; one rebuild() backing
  startListening/stop/startMic/stopMic/reconfigure/setCaptureChannels.
- IOSAudioRouter: 7 presets -> 4 (Voice Chat / Stereo Mic / Mono Mic /
  Advanced); persisted voiceProcessingEnabled + agcEnabled; setters call
  IOSAudioEngine.reconfigure() instead of audioRestart/reconcileVoicePath.
- AudioSessionManager slimmed; SessionState mic lifecycle collapsed;
  AppState wires external playback + listening at connect, stop at
  disconnect; SettingsView shows 4 presets + Advanced VPIO/AGC toggles.

No core/ABI/test changes — relies on the already-shipped external API
(test_external_pcm, test_external_playback). xcodebuild iOS device Debug
BUILD SUCCEEDED. Updates docs/voice.md §8 and PROGRESS.md.
This commit is contained in:
2026-06-23 02:45:53 +02:00
parent 7547b8e140
commit d30c4ee2f5
8 changed files with 438 additions and 414 deletions

View File

@@ -230,37 +230,46 @@ Each receiver keeps an **adaptive jitter buffer per ssrc** with **bounded-depth
in the channel's mode — stereo when the channel is stereo (real interleaved L/R, no
downmix), mono when the channel is mono — so a stereo music/screen-share channel gets
genuine stereo end-to-end. See §9 for the platform-specific loopback mechanism.
- **iOS mic capture:** all iOS audio routing is driven from Swift via `AVAudioSession` by the
`IOSAudioRouter` singleton *before* the core (miniaudio) opens its device — miniaudio does
NOT touch `AVAudioSession` on iOS. Input port selection (`availableInputs`), built-in mic
orientation (`setPreferredDataSource`: front/back/top/bottom), polar patterns
(`setPreferredPolarPattern`: omni/cardioid/subcardioid/bidirectional), mic processing mode
(`.voiceChat` = Standard with AEC/AGC/HPF, or `.measurement` = Raw/Studio with all processing
off), Bluetooth mode (`.allowBluetoothHFP` HFP voice vs `.allowBluetoothA2DP` stereo output
vs neither), and stereo capture (`.stereo` polar pattern + `setPreferredInput` +
`setInputDataSource``vc_set_capture_channels`) are all set from Swift. The core then
opens whatever route AVAudioSession has established. When the user changes audio settings
mid-session, `IOSAudioRouter` suspends the core's devices (`vc_audio_suspend`), reconfigures
`AVAudioSession`, then restarts the devices (`vc_audio_restart`) so they reopen against the
new route — mirroring TeamTalk5's `closeSoundDevices`/`initSoundInputDevice`/
`initSoundOutputDevice` pattern.
- **iOS voice processing (AEC/NS/AGC) — native VPIO path.** Real iOS echo cancellation, noise
suppression and AGC are provided ONLY by Apple's **Voice-Processing I/O audio unit (VPIO)**,
*not* by the `AVAudioSession` mode alone. The core uses miniaudio's plain `RemoteIO` audio
units, which never engage VPIO — so `.voiceChat` mode by itself yields no AEC. For VPIO to
cancel echo it must own BOTH the mic capture and the remote-audio playback (it subtracts the
played-back signal from the mic), so on the AEC presets (Voice Chat / Bluetooth Headset HFP /
Wired Headset) the Swift layer runs a native `AVAudioEngine` with
`inputNode.setVoiceProcessingEnabled(true)` and the core runs in **external mode**:
- **Mic:** the MIC stream is started with `vc_stream_desc.external_feed=1`; the VPIO input tap
feeds processed mic PCM via `vc_stream_feed_pcm`. The core skips its hardware capture device
(`AudioParams.external_capture`).
- **Playback:** `vc_set_external_playback(1)` makes the core skip its hardware playback device;
a mixer-timer thread drives decode+mix on a ~20 ms cadence and delivers the FINAL mixed PCM
via `vc_set_mixed_output_sink`. The Swift engine renders that through the VPIO output, so
VPIO has its echo-cancellation reference signal.
The Stereo Mic / Studio / A2DP presets keep the miniaudio path (they want raw / stereo /
no-AEC routing that VPIO can't provide — VPIO forces mono).
- **iOS audio — one path, always external.** On iOS the core **never opens a miniaudio device**:
a single `AVAudioEngine` (`IOSAudioEngine`) drives *both* directions, and the core runs fully
external for the whole connection. This is the single most important property of the iOS audio
stack — there is no second (miniaudio) path to switch to, so a preset/route change cannot leave
one direction dropped. The single ordering rule is: `vc_set_external_playback(1)` is set **once
at connect** (before the session is activated or any remote stream arrives), and every MIC
stream is started with `vc_stream_desc.external_feed=1`.
- **core → speaker:** the core's mixer-timer thread decodes+mixes on a ~20 ms cadence and
delivers the FINAL mixed PCM via `vc_set_mixed_output_sink`; an `AVAudioSourceNode` pulls it
from a lock-free ring and renders it. This runs the whole time we are connected, so remote
audio plays even before the user joins voice (kills the "can't hear anyone" race).
- **mic → core:** when the mic is active a tap on the engine's input node converts to 48 kHz
int16 (`vc_set_capture_channels` decides mono/stereo) and calls `vc_stream_feed_pcm`.
- **iOS routing** is still driven from Swift via `AVAudioSession` by the `IOSAudioRouter`
singleton — miniaudio never touches `AVAudioSession` on iOS. Input port selection
(`availableInputs`), built-in mic orientation (`setPreferredDataSource`: front/back/top/bottom),
polar patterns (`setPreferredPolarPattern`: omni/cardioid/subcardioid/bidirectional), mic
processing mode (Standard vs `.measurement` Raw), Bluetooth mode (`.allowBluetoothHFP` HFP voice
vs `.allowBluetoothA2DP` stereo output vs neither), and stereo capture (`.stereo` polar pattern
+ `setPreferredInput` + `setInputDataSource``vc_set_capture_channels`) are all set from
Swift. Any preset / route / interruption change funnels through one deterministic, Swift-only
rebuild: `IOSAudioEngine` stops, `IOSAudioRouter.applyConfiguration()` re-applies the
`AVAudioSession`, the graph is rebuilt against the new route, and the engine restarts. No
`vc_audio_restart`/`vc_audio_suspend` dance is needed for routing (the core has no hardware
devices to reopen) — this is the spirit of TeamTalk5's "close then re-init sound devices", but
entirely inside the Swift engine.
- **iOS voice processing (AEC/NS/AGC) — native VPIO.** Real iOS echo cancellation, noise
suppression and AGC come ONLY from Apple's **Voice-Processing I/O audio unit (VPIO)**, which
`inputNode.setVoiceProcessingEnabled(true)` enables; for it to cancel echo it must own BOTH the
mic capture and the playback — which the unified engine already does. VPIO forces **mono**, so
it is engaged only when the active config wants it (`IOSAudioRouter.currentConfigUsesVoiceProcessing`:
mono + standard + non-A2DP + the user's master toggle). iOS exposes no per-stage VPIO control,
so the Advanced UI offers exactly two switches: a master **Voice Processing** (AEC + NS bundled)
and **AGC** (`isVoiceProcessingAGCEnabled`).
- **iOS presets** (`IOSAudioRouter.AudioPreset`): **Voice Chat** (VPIO mono, system output incl.
HFP/wired), **Stereo Mic** (internal stereo built-in mic regardless of output, A2DP-capable, no
VPIO), **Mono Mic** (internal mono built-in mic regardless of output, A2DP-capable, no VPIO),
and **Advanced** (every knob manual). A2DP output requires an internal-mic preset (the Bluetooth
device is output-only); the Stereo/Mono Mic presets fall back to the built-in speaker when no
external output is connected (`applyA2dpSpeakerFallback`).
- **DSP engine: see §11.** The original plan was `webrtc-audio-processing` (AEC + NS + AGC +
VAD in one tuned module, BSD-licensed) — but it has no working Windows/MSVC build upstream
(confirmed via its own issue tracker: GCC-only Meson build, MinGW support unfinished, hard