fix(audio): bound playout depth to stop voice latency ratcheting up
Latency between speakers grew to multiple seconds and only reset on rejoining voice. Root cause was the receiver playout logic, not the codec settings: the playout clock free-ran in real time while the sender omitted silence from its timestamps (and set no header flags), and the only correction snapped the clock to the *oldest* buffered frame — which could only ever add standing latency. target_depth_ms_ was computed but never enforced, so latency could only grow or reset. Fix: bound playout against the stream's leading edge (newest frame). (Re)seed to the leading edge on start/marker/starve (no prebuffer, so latency stays low), and frame-skip catch-up trims any backlog beyond target+hysteresis — the missing downward force. Hardening: sender now stamps kFlagMarker (talkspurt start) and kFlagDtx, consumed on recv for clean resync; adaptive late-drop window; EWMA outlier rejection so silence gaps/stragglers don't poison the estimate; duplicate counting and ring-underrun diagnostics. New test_jitter_depth asserts depth stays bounded (<200ms) while arrivals outrun playback. ctest --preset dev green (27/27). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -143,16 +143,28 @@ Layered, all configurable per channel:
|
||||
|
||||
## 5. Jitter buffer
|
||||
|
||||
Each receiver keeps an **adaptive jitter buffer per ssrc**.
|
||||
Each receiver keeps an **adaptive jitter buffer per ssrc** with **bounded-depth playout**
|
||||
(`core/src/audio/audio_engine.cpp` — `JitterBuffer` + `AudioEngine::on_playback`).
|
||||
|
||||
- Frames are inserted by `timestamp`; playback reads in order at the device callback rate.
|
||||
- Target depth adapts to observed network jitter between a configurable **min/max latency**
|
||||
(channel-level "stability vs latency" knob). A "low-latency" channel runs a shallow
|
||||
buffer; a "stable" channel runs deeper.
|
||||
- Late frames past the playout point are dropped; gaps are filled by FEC (if the next frame
|
||||
- **The playout clock is always bounded against the stream's *leading edge* (newest buffered
|
||||
frame), never re-synced to the oldest.** The clock free-runs at the playback hardware rate,
|
||||
while the sender omits VAD/PTT/DTX silence from its timestamps, so the two diverge across gaps
|
||||
and late joins. Two corrections keep latency bounded:
|
||||
- **(Re)seed to the leading edge** on first frame, on a talkspurt `marker`, or when the clock
|
||||
has run past the newest frame (starved after silence). No artificial prebuffer — latency
|
||||
starts as low as possible; buffered frames still play oldest-first.
|
||||
- **Frame-skip catch-up:** when the backlog grows past `target + hysteresis` (clock drift,
|
||||
bursty arrival, reordering), fast-forward the clock to leave `target` buffered and drop the
|
||||
now-stale frames. This is the downward force that prevents latency from ratcheting upward.
|
||||
- `target` is the adaptive jitter estimate (EWMA of inter-arrival gap vs. the per-frame gap),
|
||||
floored; silence gaps and reordered stragglers are rejected as outliers so they don't inflate
|
||||
it. The late-drop window tracks `target` (floored/capped at 500 ms).
|
||||
- Late frames past the playout point are dropped; gaps are filled by DRED (if the next frame
|
||||
arrived) or PLC.
|
||||
- The `marker` flag (start of talkspurt) lets the buffer resynchronize cleanly after
|
||||
silence/DTX without accumulating drift.
|
||||
- The `marker` flag (start of talkspurt) — set by the sender on the first frame after a
|
||||
transmission gap — lets the buffer reseed cleanly after silence/DTX without accumulating drift.
|
||||
- Diagnostics per stream: `packets_lost`, `duplicates`, `underruns`, `target_depth_ms`.
|
||||
|
||||
```
|
||||
incoming (out of order) ──▶ [ reorder by ts | adaptive depth ] ──▶ Opus decode ──▶ mixer
|
||||
|
||||
Reference in New Issue
Block a user