feat(ios): real echo cancellation/NR via native Voice-Processing engine

iOS "voice chat" had echo and no noise suppression: real iOS AEC/NS/AGC
come only from Apple's Voice-Processing I/O unit (VPIO), but the core
plays/captures via miniaudio's plain RemoteIO units, so .voiceChat mode
alone never engaged AEC.

Core (ABI PATCH 1->2):
- vc_set_mixed_output_sink + vc_set_external_playback. In external mode the
  AudioEngine opens no hardware playback device; a mixer-timer thread drives
  on_playback (decode+mix) on a ~20ms cadence and ships the final mix to the
  sink. start() also skips the hardware capture device when the MIC stream is
  external_feed (AudioParams.external_capture).
- New white-box test test_external_playback (drives the timer with no hw).

iOS/Swift:
- StreamDescriptor.externalFeed; VoiceCatClient.setMixedOutputSink /
  setExternalPlayback wrappers.
- IOSVoiceProcessingEngine: AVAudioEngine + setVoiceProcessingEnabled; mic
  tap -> feedPcm, mixed-sink lock-free ring -> AVAudioSourceNode (both share
  the VPIO unit so AEC has its reference signal).
- IOSAudioRouter.currentConfigUsesVoiceProcessing scopes VPIO to the AEC
  presets; SessionState join/leave + reconcileVoicePath() switch paths;
  Voice Chat defaults to speaker; Settings surfaces AEC/NS state.

Known: pending on-device verification; a few bugs to fix afterward.
This commit is contained in:
2026-06-22 02:38:01 +02:00
parent e806b698ec
commit 6c17881cc0
19 changed files with 755 additions and 4 deletions

View File

@@ -41,6 +41,10 @@ import Foundation
/// `VcPcmSinkCallback` delegate.
public typealias VoiceCatPcmSinkCallback = vc_pcm_sink_cb
/// Swift-idiomatic alias for the C `vc_mixed_output_cb` function-pointer type from `voicecat.h`
/// the external mixed-output sink used by the iOS VPIO path (see `setMixedOutputSink`).
public typealias VoiceCatMixedOutputCallback = vc_mixed_output_cb
/// The Swift wrapper around `vc_client*`. Owns the native handle for its entire lifetime;
/// `deinit` destroys it. Events and level meters are delivered on the main queue via the
/// `onEvent` / `onLevel` closures.
@@ -292,6 +296,7 @@ public final class VoiceCatClient {
desc.kind = descriptor.kind.cValue
desc.device_id = deviceIdPtr.map { UnsafePointer($0) }
desc.label = UnsafePointer(labelPtr)
desc.external_feed = descriptor.externalFeed ? 1 : 0
let r = vc_stream_start(handle, &desc, &streamId)
return (VoiceCatResult(r), streamId)
@@ -355,6 +360,26 @@ public final class VoiceCatClient {
VoiceCatResult(vc_set_pcm_sink(handle, cb, user))
}
/// External mixed-output sink (iOS VPIO) receives the FINAL mixed remote audio as int16
/// PCM on the core's mixer-timer thread when external playback is enabled. The Swift VPIO
/// renderer copies this into its ring and plays it through the voice-processing output so
/// echo cancellation has its reference signal. Pass `nil` to disable. Mirrors
/// `vc_set_mixed_output_sink`. The callback MUST NOT block or allocate.
@discardableResult
public func setMixedOutputSink(_ cb: VoiceCatMixedOutputCallback?,
user: UnsafeMutableRawPointer?) -> VoiceCatResult {
VoiceCatResult(vc_set_mixed_output_sink(handle, cb, user))
}
/// Enable/disable external-playback mode (iOS VPIO). When enabled, the core opens NO hardware
/// playback device; it drives decode+mix on a timer and delivers the final mix via
/// `setMixedOutputSink`. Apply before the engine starts, or follow with `audioRestart()` to
/// apply to a running engine. Mirrors `vc_set_external_playback`.
@discardableResult
public func setExternalPlayback(_ enabled: Bool) -> VoiceCatResult {
VoiceCatResult(vc_set_external_playback(handle, enabled ? 1 : 0))
}
@discardableResult
public func setInputMode(_ mode: VoiceCatInputMode) -> VoiceCatResult {
VoiceCatResult(vc_set_input_mode(handle, mode.cValue))