Talon/voice-cat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(audio): buffer capture frames to Opus encoder's fixed frame size

Browse Source 
AudioEngine::on_capture() was passing miniaudio's hardware callback period
(commonly 480 samples / 10 ms on WASAPI shared mode) directly to opus_encode(),
which requires exactly frame_samples_ (960 for 20 ms @ 48 kHz). The mismatch
returned OPUS_BAD_ARG and silently dropped every real mic frame, while screen
share and injected test frames happened to be correctly sized and worked fine.

Fix: accumulate PCM in a pre-allocated CaptureAccum buffer (mirroring the
existing RemoteStream::ring fix on the playback side) and only call capture_cb_
when a full frame_samples_ chunk is ready. Same pattern applied to on_loopback().

Add test_capture_frame_accumulation() to verify the accumulator fires exactly the
right number of callbacks for misaligned chunk sizes (480, 240+720, 1920 samples).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
Talon
2026-06-17 01:51:50 +02:00
parent 45d87bde67
commit 7da0a02b3a
3 changed files with 155 additions and 13 deletions
Download Patch File Download Diff File Expand all files Collapse all files

39
core/src/audio/audio_engine.h
View File

@@ -14,6 +14,7 @@
#include <algorithm>
#include <atomic>
#include <cstring>
#include <cstdint>
#include <functional>
#include <map>
@@ -168,6 +169,30 @@ class AudioEngine {
// stereo mixing end-to-end (no audio hardware needed). Same logic the real playback
// callback uses; safe to call any time after start() (no ma_device touched).
void mix_for_test(int16_t* out, uint32_t frames) { on_playback(out, frames); }
// TEST-ONLY — drives the capture-side frame accumulator directly with an explicit
// callback, bypassing capture_cb_. Call after engine.start(p) WITHOUT a capture
// callback so the real mic (if any) never touches capture_accum_ (on_capture returns
// early when capture_cb_ is null). The explicit `cb` is invoked only when a full
// frame_samples_ chunk is ready — that is the invariant under test.
void feed_capture_for_test(const int16_t* pcm, int frames, const CaptureCallback& cb) {
if (frame_samples_ <= 0 || !cb || capture_accum_.buf.empty()) return;
const int16_t* src = pcm;
auto remaining = frames;
while (remaining > 0) {
int space = frame_samples_ - capture_accum_.count;
int copy = std::min(remaining, space);
std::memcpy(capture_accum_.buf.data() + capture_accum_.count, src,
static_cast<size_t>(copy) * sizeof(int16_t));
capture_accum_.count += copy;
src += copy;
remaining -= copy;
if (capture_accum_.count == frame_samples_) {
cb(0, capture_accum_.buf.data(), frame_samples_);
capture_accum_.count = 0;
}
}
}
#endif
private:
@@ -199,6 +224,20 @@ class AudioEngine {
CaptureCallback capture_cb_;
std::atomic<bool> running_{false};
// Capture-side frame accumulators: miniaudio fires the capture (and loopback) callback at
// whatever period the hardware/driver chooses — commonly 480 samples (10 ms) on WASAPI
// shared mode, while the Opus encoder requires exactly frame_samples_ per call (960 for
// 20 ms @ 48 kHz). Accumulate incoming PCM until a full frame is ready, then call
// capture_cb_. This mirrors the RemoteStream::ring fix on the playback side. Both
// accumulators are pre-allocated once in start(); never resized from the RT callback
// thread (satisfies architecture.md §3 — no allocation on RT threads).
struct CaptureAccum {
std::vector<int16_t> buf; // pre-sized to frame_samples_ in start()
int count = 0;
};
CaptureAccum capture_accum_; // mic / real capture device (on_capture)
CaptureAccum loopback_accum_; // screen-audio WASAPI loopback (on_loopback)
// Inject ring(s): stores raw int16 PCM written by inject_capture(), one ring per local
// stream kind so e.g. MIC and SCREEN_AUDIO can each be fed independently in tests.
// The encode thread reads from these (no real capture device needed in tests).