feat(ios): real echo cancellation/NR via native Voice-Processing engine
iOS "voice chat" had echo and no noise suppression: real iOS AEC/NS/AGC come only from Apple's Voice-Processing I/O unit (VPIO), but the core plays/captures via miniaudio's plain RemoteIO units, so .voiceChat mode alone never engaged AEC. Core (ABI PATCH 1->2): - vc_set_mixed_output_sink + vc_set_external_playback. In external mode the AudioEngine opens no hardware playback device; a mixer-timer thread drives on_playback (decode+mix) on a ~20ms cadence and ships the final mix to the sink. start() also skips the hardware capture device when the MIC stream is external_feed (AudioParams.external_capture). - New white-box test test_external_playback (drives the timer with no hw). iOS/Swift: - StreamDescriptor.externalFeed; VoiceCatClient.setMixedOutputSink / setExternalPlayback wrappers. - IOSVoiceProcessingEngine: AVAudioEngine + setVoiceProcessingEnabled; mic tap -> feedPcm, mixed-sink lock-free ring -> AVAudioSourceNode (both share the VPIO unit so AEC has its reference signal). - IOSAudioRouter.currentConfigUsesVoiceProcessing scopes VPIO to the AEC presets; SessionState join/leave + reconcileVoicePath() switch paths; Voice Chat defaults to speaker; Settings surfaces AEC/NS state. Known: pending on-device verification; a few bugs to fix afterward.
This commit is contained in:
@@ -41,6 +41,10 @@ import Foundation
|
||||
/// `VcPcmSinkCallback` delegate.
|
||||
public typealias VoiceCatPcmSinkCallback = vc_pcm_sink_cb
|
||||
|
||||
/// Swift-idiomatic alias for the C `vc_mixed_output_cb` function-pointer type from `voicecat.h`
|
||||
/// — the external mixed-output sink used by the iOS VPIO path (see `setMixedOutputSink`).
|
||||
public typealias VoiceCatMixedOutputCallback = vc_mixed_output_cb
|
||||
|
||||
/// The Swift wrapper around `vc_client*`. Owns the native handle for its entire lifetime;
|
||||
/// `deinit` destroys it. Events and level meters are delivered on the main queue via the
|
||||
/// `onEvent` / `onLevel` closures.
|
||||
@@ -292,6 +296,7 @@ public final class VoiceCatClient {
|
||||
desc.kind = descriptor.kind.cValue
|
||||
desc.device_id = deviceIdPtr.map { UnsafePointer($0) }
|
||||
desc.label = UnsafePointer(labelPtr)
|
||||
desc.external_feed = descriptor.externalFeed ? 1 : 0
|
||||
|
||||
let r = vc_stream_start(handle, &desc, &streamId)
|
||||
return (VoiceCatResult(r), streamId)
|
||||
@@ -355,6 +360,26 @@ public final class VoiceCatClient {
|
||||
VoiceCatResult(vc_set_pcm_sink(handle, cb, user))
|
||||
}
|
||||
|
||||
/// External mixed-output sink (iOS VPIO) — receives the FINAL mixed remote audio as int16
|
||||
/// PCM on the core's mixer-timer thread when external playback is enabled. The Swift VPIO
|
||||
/// renderer copies this into its ring and plays it through the voice-processing output so
|
||||
/// echo cancellation has its reference signal. Pass `nil` to disable. Mirrors
|
||||
/// `vc_set_mixed_output_sink`. The callback MUST NOT block or allocate.
|
||||
@discardableResult
|
||||
public func setMixedOutputSink(_ cb: VoiceCatMixedOutputCallback?,
|
||||
user: UnsafeMutableRawPointer?) -> VoiceCatResult {
|
||||
VoiceCatResult(vc_set_mixed_output_sink(handle, cb, user))
|
||||
}
|
||||
|
||||
/// Enable/disable external-playback mode (iOS VPIO). When enabled, the core opens NO hardware
|
||||
/// playback device; it drives decode+mix on a timer and delivers the final mix via
|
||||
/// `setMixedOutputSink`. Apply before the engine starts, or follow with `audioRestart()` to
|
||||
/// apply to a running engine. Mirrors `vc_set_external_playback`.
|
||||
@discardableResult
|
||||
public func setExternalPlayback(_ enabled: Bool) -> VoiceCatResult {
|
||||
VoiceCatResult(vc_set_external_playback(handle, enabled ? 1 : 0))
|
||||
}
|
||||
|
||||
@discardableResult
|
||||
public func setInputMode(_ mode: VoiceCatInputMode) -> VoiceCatResult {
|
||||
VoiceCatResult(vc_set_input_mode(handle, mode.cValue))
|
||||
|
||||
Reference in New Issue
Block a user