feat(ios): real echo cancellation/NR via native Voice-Processing engine
iOS "voice chat" had echo and no noise suppression: real iOS AEC/NS/AGC come only from Apple's Voice-Processing I/O unit (VPIO), but the core plays/captures via miniaudio's plain RemoteIO units, so .voiceChat mode alone never engaged AEC. Core (ABI PATCH 1->2): - vc_set_mixed_output_sink + vc_set_external_playback. In external mode the AudioEngine opens no hardware playback device; a mixer-timer thread drives on_playback (decode+mix) on a ~20ms cadence and ships the final mix to the sink. start() also skips the hardware capture device when the MIC stream is external_feed (AudioParams.external_capture). - New white-box test test_external_playback (drives the timer with no hw). iOS/Swift: - StreamDescriptor.externalFeed; VoiceCatClient.setMixedOutputSink / setExternalPlayback wrappers. - IOSVoiceProcessingEngine: AVAudioEngine + setVoiceProcessingEnabled; mic tap -> feedPcm, mixed-sink lock-free ring -> AVAudioSourceNode (both share the VPIO unit so AEC has its reference signal). - IOSAudioRouter.currentConfigUsesVoiceProcessing scopes VPIO to the AEC presets; SessionState join/leave + reconcileVoicePath() switch paths; Voice Chat defaults to speaker; Settings surfaces AEC/NS state. Known: pending on-device verification; a few bugs to fix afterward.
This commit is contained in:
25
PROGRESS.md
25
PROGRESS.md
@@ -10,6 +10,31 @@ up instantly. Newest status at the top.
|
||||
|
||||
## ▶ Where we left off / next action
|
||||
|
||||
- **Awaiting on-device verification (2026-06-22):** **iOS real echo cancellation / noise suppression
|
||||
via native VPIO.** Root cause of "voice chat doesn't sound like a call" (echo + no NR): real iOS
|
||||
AEC/NS/AGC come only from Apple's Voice-Processing I/O unit (VPIO), but the core uses miniaudio's
|
||||
plain RemoteIO units — so `.voiceChat` mode alone never engaged AEC. Fix moves both mic capture and
|
||||
playback to a native Swift `AVAudioEngine` (`setVoiceProcessingEnabled`) on the AEC presets, with the
|
||||
core in external mode.
|
||||
- **Core (done, builds + tests green):** new ABI `vc_set_mixed_output_sink` + `vc_set_external_playback`
|
||||
(voicecat.h PATCH→2). `AudioEngine` gains a mixer-timer thread that drives `on_playback` (decode+mix)
|
||||
on a ~20 ms cadence with NO hardware playback device and ships the final mix to the mixed-output
|
||||
sink; `start()` also skips the hardware capture device when the MIC stream is `external_feed`
|
||||
(`AudioParams.external_capture`). New white-box test `test_external_playback` (23/24;
|
||||
pre-existing `external_pcm` teardown crash on Darwin 25.5 is UNRELATED — original tree crashes too).
|
||||
- **Swift (done, builds):** `VoiceCatCore` wrappers (`externalFeed` on `StreamDescriptor`,
|
||||
`setMixedOutputSink`, `setExternalPlayback`); new `IOSVoiceProcessingEngine.swift` (VPIO
|
||||
`AVAudioEngine`: mic tap→`feedPcm`, mixed-sink lock-free ring→`AVAudioSourceNode`);
|
||||
`IOSAudioRouter.currentConfigUsesVoiceProcessing` gates the path per preset; `SessionState`
|
||||
join/leave + `reconcileVoicePath()` switch between VPIO and the miniaudio path; Voice Chat defaults
|
||||
to speaker; SettingsView shows AEC/NS state. **Rebuild the xcframework** before building the app:
|
||||
`clients/apple/scripts/build-xcframework.sh --all` (new ABI symbols). `xcodebuild` iOS sim Debug
|
||||
BUILD SUCCEEDED.
|
||||
- **Next (user, on device):** two iPhones on speaker, Voice Chat preset → confirm (a) no echo, (b)
|
||||
background noise suppressed, (c) speaker output by default; then Studio preset still works (raw, no
|
||||
AEC) — i.e. VPIO is scoped to the AEC presets. Tune the mixer-timer/ring sizing if there's
|
||||
under/overrun.
|
||||
|
||||
- **Done (2026-06-21):** **Docker + Linux deployment + GitHub Actions cross-build.** Added the complete Linux server
|
||||
deployment story (the only missing platform — Windows and macOS already have native
|
||||
binaries):
|
||||
|
||||
Reference in New Issue
Block a user