feat(ios): real echo cancellation/NR via native Voice-Processing engine

iOS "voice chat" had echo and no noise suppression: real iOS AEC/NS/AGC
come only from Apple's Voice-Processing I/O unit (VPIO), but the core
plays/captures via miniaudio's plain RemoteIO units, so .voiceChat mode
alone never engaged AEC.

Core (ABI PATCH 1->2):
- vc_set_mixed_output_sink + vc_set_external_playback. In external mode the
  AudioEngine opens no hardware playback device; a mixer-timer thread drives
  on_playback (decode+mix) on a ~20ms cadence and ships the final mix to the
  sink. start() also skips the hardware capture device when the MIC stream is
  external_feed (AudioParams.external_capture).
- New white-box test test_external_playback (drives the timer with no hw).

iOS/Swift:
- StreamDescriptor.externalFeed; VoiceCatClient.setMixedOutputSink /
  setExternalPlayback wrappers.
- IOSVoiceProcessingEngine: AVAudioEngine + setVoiceProcessingEnabled; mic
  tap -> feedPcm, mixed-sink lock-free ring -> AVAudioSourceNode (both share
  the VPIO unit so AEC has its reference signal).
- IOSAudioRouter.currentConfigUsesVoiceProcessing scopes VPIO to the AEC
  presets; SessionState join/leave + reconcileVoicePath() switch paths;
  Voice Chat defaults to speaker; Settings surfaces AEC/NS state.

Known: pending on-device verification; a few bugs to fix afterward.
This commit is contained in:
2026-06-22 02:38:01 +02:00
parent e806b698ec
commit 6c17881cc0
19 changed files with 755 additions and 4 deletions

View File

@@ -46,7 +46,7 @@ extern "C" {
/* ── Version ──────────────────────────────────────────────────────────────── */
#define VOICECAT_VERSION_MAJOR 0
#define VOICECAT_VERSION_MINOR 0
#define VOICECAT_VERSION_PATCH 1
#define VOICECAT_VERSION_PATCH 2 /* +vc_set_mixed_output_sink / vc_set_external_playback (iOS VPIO) */
/* The control-protocol version this build speaks (docs/protocol.md §4).
* v2 widened the UDP voice frame seq field u16 → u64 (docs/voice.md §2); a v2 server
@@ -460,6 +460,43 @@ typedef void (*vc_pcm_sink_cb)(void* user, uint32_t user_id, uint32_t stream_id,
uint32_t channels, uint32_t sample_rate);
VC_API vc_result vc_set_pcm_sink(vc_client* c, vc_pcm_sink_cb cb, void* user);
/* ── External playback (iOS VPIO / echo cancellation) ───────────────────────────
* On iOS, real echo cancellation + noise suppression + AGC are provided ONLY by Apple's
* Voice-Processing I/O audio unit (VPIO), which the Swift AVAudioEngine layer owns. For VPIO
* to cancel echo, the remote-audio playback must go through the SAME VPIO unit as the mic
* capture (VPIO subtracts the played-back signal from the mic). So in that topology the core
* must NOT open/drive its own hardware playback device — its output would bypass VPIO, giving
* it no reference signal and producing echo. Instead, enable external playback: the core keeps
* decoding + mixing every remote stream on a steady ~20 ms cadence and delivers the FINAL
* MIXED PCM (post output-volume, all streams summed) to this sink, which the Swift layer
* renders through the VPIO output.
*
* cb(user, pcm, samples_per_channel, channels, sample_rate)
*
* pcm : final mixed int16 PCM, interleaved when channels == 2.
* samples_per_channel : samples per channel for this block (960 @ 20 ms / 48 kHz).
* channels : the engine's playback channel count (2 = stereo).
* sample_rate : always 48000.
*
* The callback fires on the core's mixer-timer thread (NOT a hardware audio thread). It fires
* steadily even with no remote streams (a silent block), so the renderer has a continuous
* clock. The callback MUST NOT block, lock, or allocate — copy into a lock-free ring and
* return. Independent of vc_set_pcm_sink (the per-stream tap), which still works. Pass cb=NULL
* to disable (default: disabled). */
typedef void (*vc_mixed_output_cb)(void* user, const int16_t* pcm,
size_t samples_per_channel, uint32_t channels,
uint32_t sample_rate);
VC_API vc_result vc_set_mixed_output_sink(vc_client* c, vc_mixed_output_cb cb, void* user);
/* Enable/disable external-playback mode (default: disabled = normal hardware playback). When
* enabled, the core does NOT open a hardware playback device; decode+mix runs on an internal
* ~20 ms timer and the result is delivered via vc_set_mixed_output_sink. To also bypass the
* hardware mic (feeding VPIO-processed mic PCM instead), start the MIC stream with
* vc_stream_desc.external_feed=1 and push frames via vc_stream_feed_pcm — the core then skips
* the hardware capture device too. Apply BEFORE the engine starts, or follow with
* vc_audio_restart() to apply to a running engine. `enable` is a bool (0/1). */
VC_API vc_result vc_set_external_playback(vc_client* c, int enable);
/* ── Text ─────────────────────────────────────────────────────────────────── */
VC_API vc_result vc_send_text(vc_client* c, vc_text_scope scope, uint32_t target_id,
const char* utf8);