fix(ios): stop core opening a second mic device — dual capture/crackle
On iOS the remote end heard the mic twice and crackly (BT headset + internal mic in Voice Chat; mono + stereo copies of the internal mic in Stereo Mic). Root cause is an io-thread ordering race. `ensure_audio_running()` only set `external_capture` when a MIC stream already existed, but it also runs from `sync_remote_streams` on the post-auth `ServerStateSnapshot` — before the user joins voice. With `external_playback_` still false and no MIC stream, `AudioEngine::start()` opened a real miniaudio capture device that stayed open all session (later calls early-return on running()), racing the AVAudioEngine input tap fed via `vc_stream_feed_pcm`. `on_capture_frame` then encoded+sent both paths — the mic transmitted twice, the two unsynchronized capture clocks producing the crackle. - core (ensure_audio_running): force `external_capture = true` whenever `external_playback_` is set, so iOS unified mode never opens a hardware capture device. No-op on desktop. - ios (AppState): move `setExternalPlayback(true)` before `connect()`, so the flag is set before the io thread processes any message — closing the race. Verified on device: remote end hears the iOS mic once and clean in both Voice Chat (+ BT) and Stereo Mic.
This commit is contained in:
38
PROGRESS.md
38
PROGRESS.md
@@ -26,6 +26,44 @@ up instantly. Newest status at the top.
|
||||
send cushion could be reduced or removed. Shared-core change → add a test and re-verify
|
||||
desktop↔desktop stays low-latency (steady sender ⇒ ~0 arrival jitter ⇒ no regression).
|
||||
|
||||
- **Done (2026-06-23):** **Fixed iOS dual-stream / crackly mic — core opened a second
|
||||
(miniaudio) capture device alongside the AVAudioEngine tap.** Symptom: with two clients in
|
||||
a channel, the remote end heard the iOS mic **twice** and crackly. With Voice Chat + a BT
|
||||
headset, both the BT mic and the internal mic were captured; with Stereo Mic, both a mono
|
||||
and a stereo copy of the internal mic were sent simultaneously. Root cause is a timing gap
|
||||
in `vc_client::ensure_audio_running()` (`core/src/core/client.cpp`): `external_capture` was
|
||||
only set when a MIC stream already existed, but `ensure_audio_running` is also called from
|
||||
`sync_remote_streams` (triggered by the post-auth `ServerStateSnapshot`) **before** the user
|
||||
joins voice — so with no MIC stream, `external_capture` stayed `false` and
|
||||
`AudioEngine::start()` opened a real miniaudio capture device. Later the user joined voice →
|
||||
`IOSAudioEngine.startMic` installed the AVAudioEngine input tap → `feedPcm` →
|
||||
`inject_capture` → `on_capture_frame`. The miniaudio device was still open (the engine was
|
||||
already `running()`, so the later `ensure_audio_running` early-returned and never applied
|
||||
`external_feed`), and `on_capture_frame` encodes+sends every frame with **no deduplication**
|
||||
→ the mic was sent twice. The two unsynchronized capture clocks interleaving in the encoder
|
||||
is the crackle; the mono miniaudio device + stereo AVAudioEngine tap is the "mono and stereo
|
||||
at the same time" on Stereo Mic.
|
||||
- **Fix 1 (core, `core/src/core/client.cpp:ensure_audio_running`):** force
|
||||
`p.external_capture = true` whenever `external_playback_` is set. In iOS unified mode the
|
||||
core must never open a hardware capture device — the AVAudioEngine owns the only mic path.
|
||||
No-op on desktop (`external_playback_` is never set there).
|
||||
- **Fix 2 (iOS, `clients/apple/iOS/VoiceCatiOS/AppState.swift`):** move
|
||||
`client.setExternalPlayback(true)` from the `authResult` handler to **before**
|
||||
`client.connect(...)`. The server sends `AuthResult` immediately followed by
|
||||
`ServerStateSnapshot`; `handle_server_state` runs `ensure_audio_running` on the io thread
|
||||
before the main thread drains `authResult`, so setting the flag post-auth raced. Setting it
|
||||
pre-connect guarantees `external_playback_` is true before any message is processed —
|
||||
eliminating the playback-device race too (the mixer timer + AVAudioEngine playback path +
|
||||
VPIO AEC reference are correct from the first frame).
|
||||
- **Verify:** `cmake --build --preset dev` clean; `ctest --preset dev` = 24/28 — the 4
|
||||
failures (`vad_ptt_devices`, `external_pcm`, `frame_ms_reframe`, `channel_samplerate`) are
|
||||
a **pre-existing** teardown `mutex lock failed` race, reproduced identically with the
|
||||
changes stashed. `external_playback` (the one test exercising this code path) **passes**.
|
||||
No xcframework rebuild needed (no new symbols). **Next (manual, on device):** two clients
|
||||
in a channel — Voice Chat + BT, and Stereo Mic — confirm the remote end hears the iOS mic
|
||||
once, clean (no duplicate, no crackle); confirm the iOS user hears the remote user cleanly
|
||||
with AEC working in Voice Chat.
|
||||
|
||||
- **Done (2026-06-23):** **Fixed iOS mic flutter / crackle / octave-up.** The iOS mic was
|
||||
unusable: a consistent ~40–60 ms flutter with volume fade ("talking through a slow fan") on
|
||||
every preset. Root cause: the core sends each captured frame **synchronously**
|
||||
|
||||
@@ -71,6 +71,15 @@ final class AppState {
|
||||
client.onEvent = { [weak self] ev in
|
||||
Task { @MainActor [weak self] in self?.handleConnectEvent(ev, server: server) }
|
||||
}
|
||||
// Put the core into external-playback mode BEFORE connect, so the flag is set on the
|
||||
// io thread before any message is processed. The server sends AuthResult immediately
|
||||
// followed by ServerStateSnapshot; handle_server_state runs ensure_audio_running() on
|
||||
// the io thread, and if external_playback_ were still false at that point the core
|
||||
// would open a hardware miniaudio playback+capture device (see the matching fix in
|
||||
// vc_client::ensure_audio_running). Setting it here — before connect — guarantees the
|
||||
// unified external path is in effect from the first frame. setExternalPlayback only
|
||||
// flips an atomic + forwards to the engine's setter; both are safe pre-connect.
|
||||
client.setExternalPlayback(true)
|
||||
client.connect(host: server.host, port: server.port)
|
||||
|
||||
// Auth is queued immediately — the core serialises it behind TLS + TOFU.
|
||||
@@ -161,12 +170,10 @@ final class AppState {
|
||||
self.session = newSession
|
||||
EventFeedback.shared.play(.login)
|
||||
EventFeedback.shared.speak("Connected")
|
||||
// Put the core into external-playback mode ONCE, now, before the session is
|
||||
// activated or any remote stream can arrive — so the core never opens a miniaudio
|
||||
// device on iOS (the single ordering rule of the unified audio path). Then activate
|
||||
// the session and start the engine in listening mode so remote audio plays the
|
||||
// moment someone talks, even before we join voice (no "can't hear anyone").
|
||||
client.setExternalPlayback(true)
|
||||
// External-playback mode was enabled before connect() so the core never opens a
|
||||
// miniaudio device on iOS (the single ordering rule of the unified audio path).
|
||||
// Now activate the session and start the engine in listening mode so remote audio
|
||||
// plays the moment someone talks, even before we join voice (no "can't hear anyone").
|
||||
do {
|
||||
try AudioSessionManager.shared.ensureSessionActive()
|
||||
} catch {
|
||||
|
||||
@@ -1184,6 +1184,16 @@ void vc_client::ensure_audio_running() {
|
||||
p.external_capture = it->second.external_feed;
|
||||
}
|
||||
}
|
||||
// iOS unified mode (external_playback): capture is ALWAYS external — the Swift AVAudioEngine
|
||||
// owns the only mic path and feeds via vc_stream_feed_pcm. Force external_capture so start()
|
||||
// never opens a hardware mic device, even when ensure_audio_running runs before a MIC stream
|
||||
// exists (a remote stream arriving first — e.g. the post-auth ServerStateSnapshot — starts
|
||||
// the engine for playback). Without this the core opens a miniaudio capture device that races
|
||||
// the AVAudioEngine tap → dual mic capture → duplicated, crackly audio on the remote end.
|
||||
// No-op on desktop, where external_playback_ is never set.
|
||||
if (external_playback_.load(std::memory_order_acquire)) {
|
||||
p.external_capture = true;
|
||||
}
|
||||
// iOS VPIO: skip the hardware playback device and drive the mixer on a timer, delivering the
|
||||
// final mix to the mixed-output sink for the Swift VPIO renderer (vc_set_external_playback).
|
||||
audio_engine_.set_external_playback(external_playback_.load(std::memory_order_acquire));
|
||||
|
||||
Reference in New Issue
Block a user