fix(audio): seed/re-sync playout clock so VAD/PTT gaps don't silence playback

RemoteStream::playout_ts was seeded to 0 and only advanced inside the
decode loop (including on every PLC iteration), so it free-ran at ~1x
wall-clock regardless of whether the sender was transmitting. The
sender's frame timestamps only advance while it actually sends (the
VAD/PTT gate returns before ls.timestamp += samples). Across a late join
or any VAD/PTT silence gap the two clocks diverged without bound; once
past the jitter buffer's 500 ms late-drop window every real frame was
dropped-as-late (clock ahead) or never-due (clock behind) -> permanent
silence, while the talk indicator (driven by push_recv_frame, independent
of the jitter buffer) stayed lit.

Add JitterBuffer::peek_front_ts() (try-lock, RT-safe) and seed/re-sync
playout_ts to the earliest buffered frame on the first frame and whenever
it has drifted past +/-200/500 ms. This seeds startup and recovers after
every silence gap.

New regression test test_playout_resync free-runs the clock ~2 s past the
drop window, pushes a ts=0 frame, and asserts audible output: fails
(energy=0) without the fix, passes with it. ctest --preset m1-dev: 14/14.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-17 12:24:39 +02:00
parent 7da0a02b3a
commit a2f159e971
4 changed files with 127 additions and 0 deletions

View File

@@ -229,6 +229,68 @@ static void test_stereo_mix() {
engine.stop();
std::printf("test_stereo_mix: ok (total_diff=%lld)\n", static_cast<long long>(total_diff));
}
// ── 4b. Playout-clock re-sync after a late join / silence gap ────────────────────
// Regression for the "talk indicator lit, no audio" bug: the playout clock free-runs (it
// advances every callback via PLC), while the sender's frame timestamps only advance while it
// is actually transmitting. After a silence gap or a late join the clock drifts past the jitter
// buffer's 500 ms late-drop window, so every real frame is dropped-as-late and the stream is
// permanently silent. on_playback must re-seed the clock to the earliest buffered frame.
static void test_playout_resync() {
voicecat::audio::AudioEngine engine;
voicecat::audio::AudioParams p;
p.sample_rate = 48000;
p.capture_channels = 1;
p.playback_channels = 2;
p.frame_ms = 20;
CHECK(engine.start(p));
voicecat::codec::OpusParams mono_params; // mono = the mic path
mono_params.stereo = false;
int frame_samples = voicecat::codec::opus_frame_samples(mono_params);
voicecat::codec::OpusEncoder enc;
CHECK(enc.init(mono_params));
std::vector<int16_t> sine(static_cast<size_t>(frame_samples));
for (int i = 0; i < frame_samples; ++i) {
float t = static_cast<float>(i) / 48000.0f;
sine[i] = static_cast<int16_t>(std::sin(2.0f * 3.14159265f * 440.0f * t) * 20000.0f);
}
uint8_t opus_buf[1500];
int opus_len = enc.encode(sine.data(), frame_samples, opus_buf, sizeof(opus_buf));
CHECK(opus_len > 0);
engine.init_recv_stream(/*ssrc=*/2, mono_params);
std::vector<int16_t> out(static_cast<size_t>(frame_samples) * 2, 0);
// Free-run the playout clock with an empty jitter buffer (PLC every callback) far past the
// 500 ms late-drop window — this is what a silence gap / late join does in the field.
for (int i = 0; i < 100; ++i) // ~100 frames @ 20 ms = ~2 s, well past 500 ms
engine.mix_for_test(out.data(), static_cast<uint32_t>(frame_samples));
// Now a real frame arrives carrying a timestamp far behind the free-run clock. Without the
// re-sync it is dropped-as-late and playback stays silent; with it the clock snaps back and
// the frame is decoded and mixed.
voicecat::audio::JitterBuffer::Frame f;
f.seq = 0;
f.timestamp = 0; // stream-relative start, now far behind the drifted playout clock
f.fec_present = false;
f.payload.assign(opus_buf, opus_buf + opus_len);
engine.push_recv_frame(2, std::move(f));
std::fill(out.begin(), out.end(), 0);
engine.mix_for_test(out.data(), static_cast<uint32_t>(frame_samples));
int64_t energy = 0;
for (int16_t s : out) energy += std::abs(static_cast<int>(s));
CHECK(energy > static_cast<int64_t>(frame_samples) * 1000); // audible, not PLC silence
engine.remove_stream(2);
engine.stop();
std::printf("test_playout_resync: ok (energy=%lld)\n", static_cast<long long>(energy));
}
#endif // VOICECAT_HAS_AUDIO && VOICECAT_HAS_OPUS
// ── 5. Capture-frame accumulation (white-box, no audio hardware needed) ──────────
@@ -439,6 +501,7 @@ int main() {
test_device_enumeration();
#if defined(VOICECAT_HAS_AUDIO) && defined(VOICECAT_HAS_OPUS)
test_stereo_mix();
test_playout_resync();
#endif
#ifdef VOICECAT_HAS_AUDIO
test_capture_frame_accumulation();