fix(audio): decouple Opus decode cadence from playback callback period

on_playback() was passing miniaudio's hardware playback-callback frame
count to opus_decode()'s max_samples, instead of the decoder's fixed
frame size (960 samples @ 20ms/48kHz). Since real packets decode to
more samples than the (often smaller, e.g. ~480 on default low-latency
WASAPI) hardware period, opus_decode returned OPUS_BUFFER_TOO_SMALL on
nearly every callback -- packets were received/decrypted/jitter-buffered
correctly but never decoded into audible PCM. Result: control-plane
events and VAD worked, but zero audio in headphones.

mix_for_test()'s white-box test masked this since it always called
on_playback with frames == frame_samples, the one case where the bug
is invisible.

Fix: RemoteStream gained a small ring buffer (init_ring/push_ring/
pop_ring) that decouples decode cadence from playback-callback cadence.
on_playback now tops the ring up by decoding whole Opus frames (always
decoder.frame_samples(), never the hardware frame count) and drains
exactly what the callback asks for, silence-padding (PLC) on underrun.

Side effect: also fixes playout_ts, which was advancing by the wrong
unit (hardware frames instead of decoded samples) -- it now tracks
correctly against jitter-buffer timestamps.

ctest --test-dir build/m1-dev: 12/12 green.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
2026-06-16 17:39:49 +02:00
parent 845f995826
commit 5be869c61a
3 changed files with 123 additions and 35 deletions

View File

@@ -12,6 +12,7 @@
#ifndef VOICECAT_AUDIO_AUDIO_ENGINE_H
#define VOICECAT_AUDIO_AUDIO_ENGINE_H
#include <algorithm>
#include <atomic>
#include <cstdint>
#include <functional>
@@ -230,6 +231,59 @@ class AudioEngine {
// (already off the real-time audio thread), polled by poll_talk_transitions().
std::atomic<int64_t> last_voice_ms{0};
bool talking = false;
// ── Decode/playback decoupling ring ─────────────────────────────────────
// opus_decode() must be called with max_samples == the encoder's fixed frame size
// (decoder.frame_samples(), e.g. 960 @ 20ms/48kHz) — that's a property of the bitstream,
// not a choice. miniaudio's playback callback period is a *separate*, independently
// chosen value (often smaller, e.g. ~480 @ low-latency WASAPI defaults) and must never
// be passed to opus_decode as max_samples (doing so made decode fail basically every
// callback — silent playback bug, fixed by this ring). on_playback() tops this ring up
// by decoding whole Opus frames (decoder's channel count) and drains exactly the
// hardware-requested sample count from it each callback, padding with silence (PLC) on
// underrun. Sized once in init_ring() (called off the audio thread); never resized from
// on_playback (real-time rule).
std::vector<int16_t> ring; // capacity = (frame_samples * 8) frames * ring_channels
size_t ring_channels = 1;
size_t ring_head = 0; // next frame (sample-per-channel) to read
size_t ring_count = 0; // buffered frames (samples-per-channel) ready
std::vector<int16_t> decode_scratch; // pre-sized: frame_samples * ring_channels
void init_ring(int channels, int frame_samples) {
ring_channels = static_cast<size_t>(std::max(1, channels));
size_t cap_frames = static_cast<size_t>(std::max(1, frame_samples)) * 8; // ~160ms @20ms frames
ring.assign(cap_frames * ring_channels, 0);
ring_head = 0;
ring_count = 0;
decode_scratch.assign(static_cast<size_t>(std::max(1, frame_samples)) * ring_channels, 0);
}
// Appends `n_frames` samples-per-channel (ring_channels each) from `pcm`. Drops the
// tail (rather than overwriting unread data) if the ring is unexpectedly full — should
// not happen with the generous 8x sizing above.
void push_ring(const int16_t* pcm, size_t n_frames) {
if (ring.empty() || ring_channels == 0) return;
size_t cap_frames = ring.size() / ring_channels;
for (size_t i = 0; i < n_frames; ++i) {
if (ring_count >= cap_frames) return;
size_t widx = (ring_head + ring_count) % cap_frames;
for (size_t c = 0; c < ring_channels; ++c)
ring[widx * ring_channels + c] = pcm[i * ring_channels + c];
++ring_count;
}
}
// Pops one frame (sample-per-channel) into `out` (sized `out_channels`, zero-filled
// first — covers both a fully-drained ring and ring_channels < out_channels).
void pop_ring(int16_t* out, size_t out_channels) {
for (size_t c = 0; c < out_channels; ++c) out[c] = 0;
if (ring_count == 0 || ring.empty() || ring_channels == 0) return;
size_t cap_frames = ring.size() / ring_channels;
for (size_t c = 0; c < ring_channels && c < out_channels; ++c)
out[c] = ring[ring_head * ring_channels + c];
ring_head = (ring_head + 1) % cap_frames;
--ring_count;
}
};
mutable std::mutex streams_mu_;
std::unordered_map<uint32_t, RemoteStream> streams_;