feat(ios): real echo cancellation/NR via native Voice-Processing engine
iOS "voice chat" had echo and no noise suppression: real iOS AEC/NS/AGC come only from Apple's Voice-Processing I/O unit (VPIO), but the core plays/captures via miniaudio's plain RemoteIO units, so .voiceChat mode alone never engaged AEC. Core (ABI PATCH 1->2): - vc_set_mixed_output_sink + vc_set_external_playback. In external mode the AudioEngine opens no hardware playback device; a mixer-timer thread drives on_playback (decode+mix) on a ~20ms cadence and ships the final mix to the sink. start() also skips the hardware capture device when the MIC stream is external_feed (AudioParams.external_capture). - New white-box test test_external_playback (drives the timer with no hw). iOS/Swift: - StreamDescriptor.externalFeed; VoiceCatClient.setMixedOutputSink / setExternalPlayback wrappers. - IOSVoiceProcessingEngine: AVAudioEngine + setVoiceProcessingEnabled; mic tap -> feedPcm, mixed-sink lock-free ring -> AVAudioSourceNode (both share the VPIO unit so AEC has its reference signal). - IOSAudioRouter.currentConfigUsesVoiceProcessing scopes VPIO to the AEC presets; SessionState join/leave + reconcileVoicePath() switch paths; Voice Chat defaults to speaker; Settings surfaces AEC/NS state. Known: pending on-device verification; a few bugs to fix afterward.
This commit is contained in:
@@ -46,7 +46,7 @@ extern "C" {
|
||||
/* ── Version ──────────────────────────────────────────────────────────────── */
|
||||
#define VOICECAT_VERSION_MAJOR 0
|
||||
#define VOICECAT_VERSION_MINOR 0
|
||||
#define VOICECAT_VERSION_PATCH 1
|
||||
#define VOICECAT_VERSION_PATCH 2 /* +vc_set_mixed_output_sink / vc_set_external_playback (iOS VPIO) */
|
||||
|
||||
/* The control-protocol version this build speaks (docs/protocol.md §4).
|
||||
* v2 widened the UDP voice frame seq field u16 → u64 (docs/voice.md §2); a v2 server
|
||||
@@ -460,6 +460,43 @@ typedef void (*vc_pcm_sink_cb)(void* user, uint32_t user_id, uint32_t stream_id,
|
||||
uint32_t channels, uint32_t sample_rate);
|
||||
VC_API vc_result vc_set_pcm_sink(vc_client* c, vc_pcm_sink_cb cb, void* user);
|
||||
|
||||
/* ── External playback (iOS VPIO / echo cancellation) ───────────────────────────
|
||||
* On iOS, real echo cancellation + noise suppression + AGC are provided ONLY by Apple's
|
||||
* Voice-Processing I/O audio unit (VPIO), which the Swift AVAudioEngine layer owns. For VPIO
|
||||
* to cancel echo, the remote-audio playback must go through the SAME VPIO unit as the mic
|
||||
* capture (VPIO subtracts the played-back signal from the mic). So in that topology the core
|
||||
* must NOT open/drive its own hardware playback device — its output would bypass VPIO, giving
|
||||
* it no reference signal and producing echo. Instead, enable external playback: the core keeps
|
||||
* decoding + mixing every remote stream on a steady ~20 ms cadence and delivers the FINAL
|
||||
* MIXED PCM (post output-volume, all streams summed) to this sink, which the Swift layer
|
||||
* renders through the VPIO output.
|
||||
*
|
||||
* cb(user, pcm, samples_per_channel, channels, sample_rate)
|
||||
*
|
||||
* pcm : final mixed int16 PCM, interleaved when channels == 2.
|
||||
* samples_per_channel : samples per channel for this block (960 @ 20 ms / 48 kHz).
|
||||
* channels : the engine's playback channel count (2 = stereo).
|
||||
* sample_rate : always 48000.
|
||||
*
|
||||
* The callback fires on the core's mixer-timer thread (NOT a hardware audio thread). It fires
|
||||
* steadily even with no remote streams (a silent block), so the renderer has a continuous
|
||||
* clock. The callback MUST NOT block, lock, or allocate — copy into a lock-free ring and
|
||||
* return. Independent of vc_set_pcm_sink (the per-stream tap), which still works. Pass cb=NULL
|
||||
* to disable (default: disabled). */
|
||||
typedef void (*vc_mixed_output_cb)(void* user, const int16_t* pcm,
|
||||
size_t samples_per_channel, uint32_t channels,
|
||||
uint32_t sample_rate);
|
||||
VC_API vc_result vc_set_mixed_output_sink(vc_client* c, vc_mixed_output_cb cb, void* user);
|
||||
|
||||
/* Enable/disable external-playback mode (default: disabled = normal hardware playback). When
|
||||
* enabled, the core does NOT open a hardware playback device; decode+mix runs on an internal
|
||||
* ~20 ms timer and the result is delivered via vc_set_mixed_output_sink. To also bypass the
|
||||
* hardware mic (feeding VPIO-processed mic PCM instead), start the MIC stream with
|
||||
* vc_stream_desc.external_feed=1 and push frames via vc_stream_feed_pcm — the core then skips
|
||||
* the hardware capture device too. Apply BEFORE the engine starts, or follow with
|
||||
* vc_audio_restart() to apply to a running engine. `enable` is a bool (0/1). */
|
||||
VC_API vc_result vc_set_external_playback(vc_client* c, int enable);
|
||||
|
||||
/* ── Text ─────────────────────────────────────────────────────────────────── */
|
||||
VC_API vc_result vc_send_text(vc_client* c, vc_text_scope scope, uint32_t target_id,
|
||||
const char* utf8);
|
||||
|
||||
Reference in New Issue
Block a user