feat(ios): real echo cancellation/NR via native Voice-Processing engine

iOS "voice chat" had echo and no noise suppression: real iOS AEC/NS/AGC
come only from Apple's Voice-Processing I/O unit (VPIO), but the core
plays/captures via miniaudio's plain RemoteIO units, so .voiceChat mode
alone never engaged AEC.

Core (ABI PATCH 1->2):
- vc_set_mixed_output_sink + vc_set_external_playback. In external mode the
  AudioEngine opens no hardware playback device; a mixer-timer thread drives
  on_playback (decode+mix) on a ~20ms cadence and ships the final mix to the
  sink. start() also skips the hardware capture device when the MIC stream is
  external_feed (AudioParams.external_capture).
- New white-box test test_external_playback (drives the timer with no hw).

iOS/Swift:
- StreamDescriptor.externalFeed; VoiceCatClient.setMixedOutputSink /
  setExternalPlayback wrappers.
- IOSVoiceProcessingEngine: AVAudioEngine + setVoiceProcessingEnabled; mic
  tap -> feedPcm, mixed-sink lock-free ring -> AVAudioSourceNode (both share
  the VPIO unit so AEC has its reference signal).
- IOSAudioRouter.currentConfigUsesVoiceProcessing scopes VPIO to the AEC
  presets; SessionState join/leave + reconcileVoicePath() switch paths;
  Voice Chat defaults to speaker; Settings surfaces AEC/NS state.

Known: pending on-device verification; a few bugs to fix afterward.
This commit is contained in:
2026-06-22 02:38:01 +02:00
parent e806b698ec
commit 6c17881cc0
19 changed files with 755 additions and 4 deletions

View File

@@ -10,6 +10,31 @@ up instantly. Newest status at the top.
## ▶ Where we left off / next action
- **Awaiting on-device verification (2026-06-22):** **iOS real echo cancellation / noise suppression
via native VPIO.** Root cause of "voice chat doesn't sound like a call" (echo + no NR): real iOS
AEC/NS/AGC come only from Apple's Voice-Processing I/O unit (VPIO), but the core uses miniaudio's
plain RemoteIO units — so `.voiceChat` mode alone never engaged AEC. Fix moves both mic capture and
playback to a native Swift `AVAudioEngine` (`setVoiceProcessingEnabled`) on the AEC presets, with the
core in external mode.
- **Core (done, builds + tests green):** new ABI `vc_set_mixed_output_sink` + `vc_set_external_playback`
(voicecat.h PATCH→2). `AudioEngine` gains a mixer-timer thread that drives `on_playback` (decode+mix)
on a ~20 ms cadence with NO hardware playback device and ships the final mix to the mixed-output
sink; `start()` also skips the hardware capture device when the MIC stream is `external_feed`
(`AudioParams.external_capture`). New white-box test `test_external_playback` (23/24;
pre-existing `external_pcm` teardown crash on Darwin 25.5 is UNRELATED — original tree crashes too).
- **Swift (done, builds):** `VoiceCatCore` wrappers (`externalFeed` on `StreamDescriptor`,
`setMixedOutputSink`, `setExternalPlayback`); new `IOSVoiceProcessingEngine.swift` (VPIO
`AVAudioEngine`: mic tap→`feedPcm`, mixed-sink lock-free ring→`AVAudioSourceNode`);
`IOSAudioRouter.currentConfigUsesVoiceProcessing` gates the path per preset; `SessionState`
join/leave + `reconcileVoicePath()` switch between VPIO and the miniaudio path; Voice Chat defaults
to speaker; SettingsView shows AEC/NS state. **Rebuild the xcframework** before building the app:
`clients/apple/scripts/build-xcframework.sh --all` (new ABI symbols). `xcodebuild` iOS sim Debug
BUILD SUCCEEDED.
- **Next (user, on device):** two iPhones on speaker, Voice Chat preset → confirm (a) no echo, (b)
background noise suppressed, (c) speaker output by default; then Studio preset still works (raw, no
AEC) — i.e. VPIO is scoped to the AEC presets. Tune the mixer-timer/ring sizing if there's
under/overrun.
- **Done (2026-06-21):** **Docker + Linux deployment + GitHub Actions cross-build.** Added the complete Linux server
deployment story (the only missing platform — Windows and macOS already have native
binaries):