fix(audio): bound playout depth to stop voice latency ratcheting up
Some checks failed
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled

Latency between speakers grew to multiple seconds and only reset on
rejoining voice. Root cause was the receiver playout logic, not the
codec settings: the playout clock free-ran in real time while the
sender omitted silence from its timestamps (and set no header flags),
and the only correction snapped the clock to the *oldest* buffered
frame — which could only ever add standing latency. target_depth_ms_
was computed but never enforced, so latency could only grow or reset.

Fix: bound playout against the stream's leading edge (newest frame).
(Re)seed to the leading edge on start/marker/starve (no prebuffer, so
latency stays low), and frame-skip catch-up trims any backlog beyond
target+hysteresis — the missing downward force.

Hardening: sender now stamps kFlagMarker (talkspurt start) and kFlagDtx,
consumed on recv for clean resync; adaptive late-drop window; EWMA
outlier rejection so silence gaps/stragglers don't poison the estimate;
duplicate counting and ring-underrun diagnostics.

New test_jitter_depth asserts depth stays bounded (<200ms) while
arrivals outrun playback. ctest --preset dev green (27/27).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-22 20:02:20 +02:00
parent e155e342f4
commit ce2035f271
10 changed files with 384 additions and 49 deletions

View File

@@ -143,16 +143,28 @@ Layered, all configurable per channel:
## 5. Jitter buffer
Each receiver keeps an **adaptive jitter buffer per ssrc**.
Each receiver keeps an **adaptive jitter buffer per ssrc** with **bounded-depth playout**
(`core/src/audio/audio_engine.cpp``JitterBuffer` + `AudioEngine::on_playback`).
- Frames are inserted by `timestamp`; playback reads in order at the device callback rate.
- Target depth adapts to observed network jitter between a configurable **min/max latency**
(channel-level "stability vs latency" knob). A "low-latency" channel runs a shallow
buffer; a "stable" channel runs deeper.
- Late frames past the playout point are dropped; gaps are filled by FEC (if the next frame
- **The playout clock is always bounded against the stream's *leading edge* (newest buffered
frame), never re-synced to the oldest.** The clock free-runs at the playback hardware rate,
while the sender omits VAD/PTT/DTX silence from its timestamps, so the two diverge across gaps
and late joins. Two corrections keep latency bounded:
- **(Re)seed to the leading edge** on first frame, on a talkspurt `marker`, or when the clock
has run past the newest frame (starved after silence). No artificial prebuffer — latency
starts as low as possible; buffered frames still play oldest-first.
- **Frame-skip catch-up:** when the backlog grows past `target + hysteresis` (clock drift,
bursty arrival, reordering), fast-forward the clock to leave `target` buffered and drop the
now-stale frames. This is the downward force that prevents latency from ratcheting upward.
- `target` is the adaptive jitter estimate (EWMA of inter-arrival gap vs. the per-frame gap),
floored; silence gaps and reordered stragglers are rejected as outliers so they don't inflate
it. The late-drop window tracks `target` (floored/capped at 500 ms).
- Late frames past the playout point are dropped; gaps are filled by DRED (if the next frame
arrived) or PLC.
- The `marker` flag (start of talkspurt) lets the buffer resynchronize cleanly after
silence/DTX without accumulating drift.
- The `marker` flag (start of talkspurt) — set by the sender on the first frame after a
transmission gap — lets the buffer reseed cleanly after silence/DTX without accumulating drift.
- Diagnostics per stream: `packets_lost`, `duplicates`, `underruns`, `target_depth_ms`.
```
incoming (out of order) ──▶ [ reorder by ts | adaptive depth ] ──▶ Opus decode ──▶ mixer