fix(audio): bound playout depth to stop voice latency ratcheting up
Latency between speakers grew to multiple seconds and only reset on rejoining voice. Root cause was the receiver playout logic, not the codec settings: the playout clock free-ran in real time while the sender omitted silence from its timestamps (and set no header flags), and the only correction snapped the clock to the *oldest* buffered frame — which could only ever add standing latency. target_depth_ms_ was computed but never enforced, so latency could only grow or reset. Fix: bound playout against the stream's leading edge (newest frame). (Re)seed to the leading edge on start/marker/starve (no prebuffer, so latency stays low), and frame-skip catch-up trims any backlog beyond target+hysteresis — the missing downward force. Hardening: sender now stamps kFlagMarker (talkspurt start) and kFlagDtx, consumed on recv for clean resync; adaptive late-drop window; EWMA outlier rejection so silence gaps/stragglers don't poison the estimate; duplicate counting and ring-underrun diagnostics. New test_jitter_depth asserts depth stays bounded (<200ms) while arrivals outrun playback. ctest --preset dev green (27/27). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
19
PROGRESS.md
19
PROGRESS.md
@@ -10,6 +10,25 @@ up instantly. Newest status at the top.
|
||||
|
||||
## ▶ Where we left off / next action
|
||||
|
||||
- **Done (2026-06-22):** **Fixed growing voice latency (jitter-buffer depth ratchet).** Symptom:
|
||||
end-to-end latency grew to multiple seconds and "drifted backward," reset only by leaving/
|
||||
rejoining voice (DTX/FEC/DRED on, 10% loss). Root cause was **not** the codec settings (10% loss
|
||||
is just an `OPUS_SET_PACKET_LOSS_PERC` encoder hint; FEC/DRED add no standing latency) but the
|
||||
receiver playout logic in `core/src/audio/audio_engine.cpp`: the playout clock free-ran in real
|
||||
time while the sender omitted silence from its timestamps and set **no header flags at all**, and
|
||||
the only correction snapped the clock to the *oldest* buffered frame (could only *add* latency) —
|
||||
with `target_depth_ms_` computed but never enforced, so latency could only grow or be reset.
|
||||
**Fix:** bounded-depth playout — (re)seed to the *leading edge* (newest frame) on start/marker/
|
||||
starve, and **frame-skip catch-up** that trims a backlog beyond `target + hysteresis` (the missing
|
||||
downward force). Plus hardening: adaptive late-drop window, talkspurt `kFlagMarker`/`kFlagDtx`
|
||||
now actually stamped by the sender (`client.cpp` send path) and consumed on recv, EWMA outlier
|
||||
rejection (silence gaps/stragglers no longer poison the estimate), duplicate counting, ring-
|
||||
underrun diagnostics (`stream_underruns`/`stream_duplicates`). New regression test
|
||||
`tests/test_jitter_depth.cpp` asserts depth stays bounded (<200 ms) while arrivals outrun playout
|
||||
for ~4 s. `ctest --preset dev` green — **27/27**. Docs: `docs/voice.md` §5 rewritten.
|
||||
- **Next (manual E2E):** two clients in a channel, DTX/FEC/DRED on — talk in alternating bursts
|
||||
for several minutes and confirm latency stays low/stable (no backward drift, no rejoin needed).
|
||||
|
||||
- **Windows done / Apple awaiting Mac build (2026-06-22):** **Event sound effects + optional
|
||||
text-to-speech for all clients.** Clients now play a cue per session event and can optionally
|
||||
speak it (TTS off by default; when on it announces joins/leaves and reads message/PM bodies).
|
||||
|
||||
Reference in New Issue
Block a user