fix(ios): pace mic feed with a prebuffer cushion to stop flutter/crackle
The iOS mic was unusable — a consistent ~40-60ms flutter + volume fade
('slow fan') on every preset. The core sends each captured frame
synchronously (no send pacer), so packet cadence == capture cadence, and
the receiver's playout keeps near-zero buffering by design (its jitter
estimate keys off the regular sender timestamp, so it's blind to arrival
jitter). That's smooth only for a steady sender (desktop miniaudio =
steady 20ms); the iOS AVAudioEngine tap delivers ~2 frames per ~40ms
callback -> bursty -> receiver underruns -> PLC fade.
Fix (iOS-only): the mic tap writes converted 48kHz int16 to an SPSC ring;
a 20ms feed pump drains it and calls feedPcm at a steady cadence. The pump
primes a small prebuffer cushion (3 frames ~60ms, self-healing up to
~120ms on underrun) before releasing, so the tap's bursts can't drain it
to empty. Never reads a partial frame (read consumes what it returns ->
partials were the crackle), and rebuilds with the current channel count
each rebuild() (a frozen count fed mono-as-stereo = octave-up on a
Stereo->Voice Chat switch).
Trade-off: ~60-120ms added mic-send latency, unavoidable when de-bursting
for a near-zero-buffer receiver. PROGRESS.md notes the proper follow-up:
make the jitter buffer measure real RFC-3550 arrival jitter so the
receiver absorbs bursts itself.
Verified: xcodebuild Debug BUILD SUCCEEDED (iOS Simulator, arm64).
This commit is contained in:
40
PROGRESS.md
40
PROGRESS.md
@@ -10,6 +10,46 @@ up instantly. Newest status at the top.
|
||||
|
||||
## ▶ Where we left off / next action
|
||||
|
||||
- **[ ] Soon — jitter buffer should measure REAL arrival jitter (RFC 3550), not sender
|
||||
timestamps.** `JitterBuffer::push` (`core/src/audio/audio_engine.cpp:84-108`) estimates
|
||||
jitter from `gap = ts - last_push_ts_`, where `ts` is the **sender's timestamp** — which is
|
||||
perfectly regular (`ls.timestamp += samples` every frame, independent of when the packet is
|
||||
actually sent). So `diff` is always ~0, `jitter_est_` stays 0, and `target_depth_ms_` is
|
||||
pinned at its ~20 ms floor. The buffer is therefore **blind to real network/arrival jitter
|
||||
and to bursty senders** — it never deepens. Combined with the playout deliberately seeding
|
||||
to near-zero depth (`on_playback`, ~line 715), the receiver tolerates only a *steady*
|
||||
sender. This is exactly why the iOS mic needed a send-side pacing cushion (below) and why
|
||||
genuine network jitter would also cause underruns. **Fix:** measure inter-arrival jitter
|
||||
the RFC 3550 way — `D = (arrival_j - arrival_i) - (ts_j - ts_i)` using a wall-clock arrival
|
||||
stamp captured in `push()` — and drive `target_depth_ms_` off that EWMA (keep the existing
|
||||
marker/silence-gap outlier rejection). Then the receiver absorbs bursts itself and the iOS
|
||||
send cushion could be reduced or removed. Shared-core change → add a test and re-verify
|
||||
desktop↔desktop stays low-latency (steady sender ⇒ ~0 arrival jitter ⇒ no regression).
|
||||
|
||||
- **Done (2026-06-23):** **Fixed iOS mic flutter / crackle / octave-up.** The iOS mic was
|
||||
unusable: a consistent ~40–60 ms flutter with volume fade ("talking through a slow fan") on
|
||||
every preset. Root cause: the core sends each captured frame **synchronously**
|
||||
(`on_capture_frame` → `encode_and_send_frame`, no send pacer), so packet cadence == capture
|
||||
cadence; and the receiver's playout keeps **near-zero buffering** by design and its jitter
|
||||
estimate is blind to arrival timing (see RFC-3550 item above). That's smooth only for a
|
||||
*steady* sender (desktop miniaudio = steady 20 ms), but the iOS `AVAudioEngine` input tap
|
||||
delivers ~2 frames per ~40 ms callback (more under VPIO) → bursty → receiver underruns → PLC
|
||||
fade.
|
||||
- **Fix (iOS-only, `clients/apple/iOS/VoiceCatiOS/IOSVoiceProcessingEngine.swift`):** the
|
||||
mic tap converts to 48 kHz int16 and writes a lock-free SPSC ring; a 20 ms feed pump
|
||||
drains it and calls `feedPcm` at a **steady** cadence so packets leave the core every
|
||||
20 ms (what the receiver expects). The pump **primes a small prebuffer cushion**
|
||||
(`PumpState.targetFrames`, 3 frames ≈ 60 ms, self-healing up to ~120 ms on underrun)
|
||||
before releasing, so the tap's bursts can't drain it to empty. Two correctness rules
|
||||
(each had bit us): never read a partial frame (`read` consumes what it returns →
|
||||
discarding partials caused crackle), and rebuild the pump with the current channel count
|
||||
every `rebuild()` (a frozen channel count fed mono-as-stereo = octave-up on a Stereo→Voice
|
||||
Chat switch). Trade-off: ~60–120 ms added mic-send latency — unavoidable when de-bursting
|
||||
for a near-zero-buffer receiver; the RFC-3550 fix above would let us shrink it.
|
||||
- **Verify:** `xcodebuild` Debug **BUILD SUCCEEDED** (iOS Simulator, arm64). Audible test
|
||||
requires a real device (simulator has no real mic route): mic should be smooth on Voice
|
||||
Chat / Mono Mic / Stereo Mic, including switching presets while live (no octave).
|
||||
|
||||
- **Done (2026-06-23):** **Fixed Apple client link failure (stale xcframework missing
|
||||
RNNoise).** Both `VoiceCatMac` and `VoiceCatiOS` failed to link with `Undefined symbols for
|
||||
architecture arm64: _rnnoise_create / _rnnoise_destroy / _rnnoise_process_frame`. Root
|
||||
|
||||
Reference in New Issue
Block a user