fix(audio): bound playout depth to stop voice latency ratcheting up
Some checks failed
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled

Latency between speakers grew to multiple seconds and only reset on
rejoining voice. Root cause was the receiver playout logic, not the
codec settings: the playout clock free-ran in real time while the
sender omitted silence from its timestamps (and set no header flags),
and the only correction snapped the clock to the *oldest* buffered
frame — which could only ever add standing latency. target_depth_ms_
was computed but never enforced, so latency could only grow or reset.

Fix: bound playout against the stream's leading edge (newest frame).
(Re)seed to the leading edge on start/marker/starve (no prebuffer, so
latency stays low), and frame-skip catch-up trims any backlog beyond
target+hysteresis — the missing downward force.

Hardening: sender now stamps kFlagMarker (talkspurt start) and kFlagDtx,
consumed on recv for clean resync; adaptive late-drop window; EWMA
outlier rejection so silence gaps/stragglers don't poison the estimate;
duplicate counting and ring-underrun diagnostics.

New test_jitter_depth asserts depth stays bounded (<200ms) while
arrivals outrun playback. ctest --preset dev green (27/27).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-22 20:02:20 +02:00
parent e155e342f4
commit ce2035f271
10 changed files with 384 additions and 49 deletions

View File

@@ -10,6 +10,25 @@ up instantly. Newest status at the top.
## ▶ Where we left off / next action
- **Done (2026-06-22):** **Fixed growing voice latency (jitter-buffer depth ratchet).** Symptom:
end-to-end latency grew to multiple seconds and "drifted backward," reset only by leaving/
rejoining voice (DTX/FEC/DRED on, 10% loss). Root cause was **not** the codec settings (10% loss
is just an `OPUS_SET_PACKET_LOSS_PERC` encoder hint; FEC/DRED add no standing latency) but the
receiver playout logic in `core/src/audio/audio_engine.cpp`: the playout clock free-ran in real
time while the sender omitted silence from its timestamps and set **no header flags at all**, and
the only correction snapped the clock to the *oldest* buffered frame (could only *add* latency) —
with `target_depth_ms_` computed but never enforced, so latency could only grow or be reset.
**Fix:** bounded-depth playout — (re)seed to the *leading edge* (newest frame) on start/marker/
starve, and **frame-skip catch-up** that trims a backlog beyond `target + hysteresis` (the missing
downward force). Plus hardening: adaptive late-drop window, talkspurt `kFlagMarker`/`kFlagDtx`
now actually stamped by the sender (`client.cpp` send path) and consumed on recv, EWMA outlier
rejection (silence gaps/stragglers no longer poison the estimate), duplicate counting, ring-
underrun diagnostics (`stream_underruns`/`stream_duplicates`). New regression test
`tests/test_jitter_depth.cpp` asserts depth stays bounded (<200 ms) while arrivals outrun playout
for ~4 s. `ctest --preset dev` green **27/27**. Docs: `docs/voice.md` §5 rewritten.
- **Next (manual E2E):** two clients in a channel, DTX/FEC/DRED on talk in alternating bursts
for several minutes and confirm latency stays low/stable (no backward drift, no rejoin needed).
- **Windows done / Apple awaiting Mac build (2026-06-22):** **Event sound effects + optional
text-to-speech for all clients.** Clients now play a cue per session event and can optionally
speak it (TTS off by default; when on it announces joins/leaves and reads message/PM bodies).