fix(audio): reframe send path to channel frame_ms; pin codec to 48 kHz
The AudioEngine capture clock is fixed at 48 kHz / 20 ms (960-sample frames), but a channel may set any Opus frame_ms (2.5..60 ms, voice.md §3) and the server enforces it unclamped. on_capture_frame handed the engine's 960-sample frame straight to an encoder configured for the channel's window: frame_ms > 20 was silently ignored, and frame_ms < 20 broke entirely (receiver sized its decode buffer too small -> OPUS_BUFFER_TOO_SMALL -> dead audio). Affected the hardware mic and vc_stream_feed_pcm alike. Reframe each captured/fed block to ls.frame_samples via a per-LocalStream accumulator (pre-sized at announce, no RT-thread alloc) before encode_and_send_frame; the 20 ms case stays a zero-copy fast path. Also pin the codec to 48 kHz in opus_params_from_audio_config — it was honoring a non-48k effective sample_rate against a 48 kHz PCM clock. New ctest frame_ms_reframe covers 40 ms (accumulate) and 10 ms (split) feed->encode->relay->decode->sink round trips. ctest --preset dev 25/25. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -428,7 +428,13 @@ VC_API vc_result vc_set_capture_channels(vc_client* c, uint32_t stream_id, uint3
|
||||
* stream must already be started (vc_stream_start). The core frames, encodes (Opus), seals
|
||||
* (AEAD), and sends (UDP) the provided samples exactly as it would mic/loopback audio.
|
||||
*
|
||||
* samples_per_channel : samples per channel (e.g. 960 for 20 ms @ 48 kHz).
|
||||
* pcm MUST be 48 kHz int16 — the core does NOT resample. (The whole audio engine runs at
|
||||
* 48 kHz; see docs/voice.md §3. A bot is responsible for resampling its source to 48 kHz.)
|
||||
*
|
||||
* samples_per_channel : samples per channel for THIS call. Any count is accepted — the core
|
||||
* buffers and re-chunks to the channel's Opus frame size (the channel's frame_ms decides
|
||||
* this: 480 @ 10 ms, 960 @ 20 ms, 1920 @ 40 ms, …). You need not match the frame size, and
|
||||
* a channel with a non-20 ms window is handled transparently.
|
||||
* channels : 1 (mono) or 2 (stereo interleaved L/R). VC_ERR_INVALID_ARG otherwise.
|
||||
*
|
||||
* Use cases: ReplayKit Broadcast Extension (iOS), ScreenCaptureKit (macOS), bots (TTS /
|
||||
|
||||
Reference in New Issue
Block a user