Talon/voice-cat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(audio): wire in-band FEC into the decoder loss path
Some checks failed
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Details
Build Linux Binaries / linux/arm64 (push) Has been cancelled
Details

Browse Source 
The encoder set OPUS_SET_INBAND_FEC, but the decoder never invoked FEC --
the loss path went DRED -> PLC, so FEC redundancy was emitted (and paid for
in bitrate) yet never consumed.

Wire FEC recovery into AudioEngine::on_playback between DRED and PLC: copy
the next buffered packet once, try DRED, else (if the stream negotiated FEC)
decode(next_pkt, ..., fec=true), else PLC. Recovery priority is now
DRED -> FEC -> PLC. Add per-stream RemoteStream::fec_enabled_, captured from
OpusParams in init_recv_stream. Docs (voice.md) updated to match.

ctest --preset dev: 27/27 green.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Talon
2026-06-22 20:18:29 +02:00
parent ce2035f271
commit 7547b8e140
4 changed files with 62 additions and 29 deletions
Download Patch File Download Diff File Expand all files Collapse all files

21
docs/voice.md
View File

@@ -125,11 +125,14 @@ Guidance baked into defaults / docs:
Layered, all configurable per channel:
1. **In-band FEC** — Opus embeds a low-bitrate copy of the previous frame; the decoder
recovers a lost packet from the *next* one (costs one frame of latency on recovery).
Tuned by `expected_packet_loss`.
1. **In-band FEC** — the encoder embeds a low-bitrate copy of the current frame in the
*next* packet (`OPUS_SET_INBAND_FEC`, redundancy scaled by `expected_packet_loss`). On a
loss, the receiver decodes that copy out of the next already-buffered packet with
`opus_decode(..., decode_fec=1)` — costing one frame of latency on recovery. Gated on the
per-stream `fec` flag; if the next packet carries no redundancy libopus yields PLC output,
so it is at worst a no-op relative to (2).
2. **PLC (packet loss concealment)** — decoder synthesizes a plausible frame for an
unrecovered loss; always on, free.
unrecovered loss; always on, free. The terminal fallback when neither DRED nor FEC applies.
3. **DTX** — sender stops transmitting during silence and sends sparse comfort-noise
updates; cuts bandwidth and is bandwidth-friendly on busy channels.
4. **DRED (Deep REDundancy, per-channel toggle)** — Opus 1.6's ML redundancy: the encoder
@@ -137,9 +140,13 @@ Layered, all configurable per channel:
default). When a packet is lost, the receiver peeks at the next already-buffered packet,
parses its DRED extension (`opus_dred_parse`), and reconstructs the lost frame with
`opus_decoder_dred_decode` — producing significantly better audio than PLC comfort noise
for single-frame gaps. Falls back silently to PLC if the next packet has not arrived yet
or if the sender did not embed DRED. Heavier CPU on the encoder (~5–10 % at 24 kbps);
minimal overhead on the decoder (parse is a fast header check on non-DRED packets).
for single-frame gaps. Heavier CPU on the encoder (~5–10 % at 24 kbps); minimal overhead
on the decoder (parse is a fast header check on non-DRED packets).
When a frame is lost, `AudioEngine::on_playback` tries these recovery paths in quality order,
falling through on failure: **DRED → in-band FEC → PLC**. DRED and FEC both need the next
packet already buffered (one frame of look-ahead); when it has not arrived yet, recovery falls
straight through to PLC.
## 5. Jitter buffer