fix(ios-audio): unify iOS audio onto one always-external AVAudioEngine
The iOS audio path was a hybrid: Voice-Chat-class presets ran a native VPIO AVAudioEngine (core external) while Stereo/Studio/A2DP presets ran the core's miniaudio devices. Nearly every "no input / no output / both" bug lived in the seam between the two paths — the lingering miniaudio capture unit fighting VPIO, the audioRestart ordering dance, the route-change "glitching" loop, stereo<->mono stickiness, and "can't hear anyone". Switching presets/routes mid-call routinely dropped a direction. Drive ALL iOS audio through one AVAudioEngine with the core fully external at all times: setExternalPlayback(1) once at connect, every MIC stream external_feed=1, mic via vc_stream_feed_pcm, playback via vc_set_mixed_output_sink (drained by an always-on AVAudioSourceNode so remote audio plays before joining voice). VPIO + AGC toggle per preset. Every preset/route/interruption change funnels through one deterministic Swift-only reconfigure (stop -> apply session config -> rebuild -> start) — no second path to hand off to, so a change can't drop a direction. - IOSVoiceProcessingEngine.swift -> IOSAudioEngine: always-on source-node playback, conditional mic tap, VPIO/AGC; one rebuild() backing startListening/stop/startMic/stopMic/reconfigure/setCaptureChannels. - IOSAudioRouter: 7 presets -> 4 (Voice Chat / Stereo Mic / Mono Mic / Advanced); persisted voiceProcessingEnabled + agcEnabled; setters call IOSAudioEngine.reconfigure() instead of audioRestart/reconcileVoicePath. - AudioSessionManager slimmed; SessionState mic lifecycle collapsed; AppState wires external playback + listening at connect, stop at disconnect; SettingsView shows 4 presets + Advanced VPIO/AGC toggles. No core/ABI/test changes — relies on the already-shipped external API (test_external_pcm, test_external_playback). xcodebuild iOS device Debug BUILD SUCCEEDED. Updates docs/voice.md §8 and PROGRESS.md.
This commit is contained in:
38
PROGRESS.md
38
PROGRESS.md
@@ -75,8 +75,42 @@ up instantly. Newest status at the top.
|
||||
is now correct. If voice still fails, watch the server's rate-limited `[media] dropped frames —
|
||||
unmapped-endpoint=…` line (NAT source-port rewrite would be the next suspect).
|
||||
|
||||
- **Awaiting on-device verification (2026-06-22):** **iOS real echo cancellation / noise suppression
|
||||
via native VPIO.** Root cause of "voice chat doesn't sound like a call" (echo + no NR): real iOS
|
||||
- **Done (2026-06-23, Swift-only — no core/ABI change; awaiting on-device verification):** **iOS audio
|
||||
stack unified — one always-external `AVAudioEngine`, miniaudio dropped on iOS.** The iOS audio path was
|
||||
a fragile hybrid: Voice-Chat-class presets ran a native VPIO `AVAudioEngine` (core external) while
|
||||
Stereo/Studio/A2DP presets ran the core's miniaudio devices. Nearly every bug lived in the seam
|
||||
(lingering miniaudio capture unit fighting VPIO, the `audioRestart` ordering dance, the route-change
|
||||
"glitching" loop, stereo↔mono stickiness, "can't hear anyone"), and switching presets/routes mid-call
|
||||
routinely dropped input, output, or both. **Fix: drive *all* iOS audio through one `AVAudioEngine` with
|
||||
the core fully external at all times** — `vc_set_external_playback(1)` once at connect, every MIC stream
|
||||
`external_feed=1`, mic via `vc_stream_feed_pcm`, playback via `vc_set_mixed_output_sink`.
|
||||
- `IOSVoiceProcessingEngine.swift` → **`IOSAudioEngine`** (same file): always-on `AVAudioSourceNode`
|
||||
playback (runs whenever connected, so remote audio plays before you join voice); conditional mic tap;
|
||||
VPIO + AGC toggled per config. One private `rebuild()` (stop → set VPIO → install tap → start) backs
|
||||
`startListening`/`stop`/`startMic`/`stopMic`/`reconfigure`/`setCaptureChannels`. Kept the `PCMRing`
|
||||
and ring-stats diagnostics.
|
||||
- `IOSAudioRouter`: presets cut from seven to **four** — Voice Chat (VPIO mono, system output),
|
||||
Stereo Mic / Mono Mic (internal built-in mic regardless of output, A2DP-capable, no VPIO), Advanced
|
||||
(manual). New persisted `voiceProcessingEnabled` (master AEC+NS) + `agcEnabled`; setters now call
|
||||
`IOSAudioEngine.reconfigure()` instead of `client.audioRestart()` + `reconcileVoicePath`. Kept the
|
||||
proven AVAudioSession recipes (category/mode/options, stereo capsule, `applyA2dpSpeakerFallback`).
|
||||
- `AudioSessionManager` slimmed (drops `client`/`activeMicStreamId`/`reconcileVoicePath`; adds
|
||||
`isActive`); interruption-end & device-change now `reconfigure()` the engine. `SessionState`
|
||||
`doStartMicStream`/`stopMicStream` collapsed to start-stream + `startMic`/`stopMic` (no
|
||||
`setExternalPlayback`/`audioRestart` toggling); `reconcileVoicePath` deleted. `AppState` sets external
|
||||
playback + `startListening` at connect, `stop()` at disconnect. `SettingsView` → four presets +
|
||||
Advanced VPIO/AGC toggles.
|
||||
- **No core/ABI/test change** — relies on the already-shipped `vc_set_external_playback` /
|
||||
`external_feed` / `vc_set_mixed_output_sink` / `vc_stream_feed_pcm` path (`test_external_pcm`,
|
||||
`test_external_playback`). `xcodebuild` iOS device Debug **BUILD SUCCEEDED**. **Rebuild the
|
||||
xcframework is NOT required** (no new symbols).
|
||||
- **Next (user, on device):** two iPhones in a channel — verify BOTH directions survive every
|
||||
transition and are never silent unless intended: Voice Chat (no echo, NR), listen-only before joining,
|
||||
join↔leave repeatedly, switch Voice Chat↔Stereo↔Mono↔Advanced *while in voice*, A2DP connect/unplug,
|
||||
wired connect/unplug, phone-call interruption + resume, screen-audio share.
|
||||
|
||||
- **Superseded by the 2026-06-23 unification above (2026-06-22):** **iOS real echo cancellation / noise
|
||||
suppression via native VPIO.** Root cause of "voice chat doesn't sound like a call" (echo + no NR): real iOS
|
||||
AEC/NS/AGC come only from Apple's Voice-Processing I/O unit (VPIO), but the core uses miniaudio's
|
||||
plain RemoteIO units — so `.voiceChat` mode alone never engaged AEC. Fix moves both mic capture and
|
||||
playback to a native Swift `AVAudioEngine` (`setVoiceProcessingEnabled`) on the AEC presets, with the
|
||||
|
||||
Reference in New Issue
Block a user