Talon/voice-cat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(audio): reframe send path to channel frame_ms; pin codec to 48 kHz
Some checks failed
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Details
Build Linux Binaries / linux/arm64 (push) Has been cancelled
Details

Browse Source 
The AudioEngine capture clock is fixed at 48 kHz / 20 ms (960-sample
frames), but a channel may set any Opus frame_ms (2.5..60 ms, voice.md
§3) and the server enforces it unclamped. on_capture_frame handed the
engine's 960-sample frame straight to an encoder configured for the
channel's window: frame_ms > 20 was silently ignored, and frame_ms < 20
broke entirely (receiver sized its decode buffer too small ->
OPUS_BUFFER_TOO_SMALL -> dead audio). Affected the hardware mic and
vc_stream_feed_pcm alike.

Reframe each captured/fed block to ls.frame_samples via a per-LocalStream
accumulator (pre-sized at announce, no RT-thread alloc) before
encode_and_send_frame; the 20 ms case stays a zero-copy fast path. Also
pin the codec to 48 kHz in opus_params_from_audio_config — it was honoring
a non-48k effective sample_rate against a 48 kHz PCM clock.

New ctest frame_ms_reframe covers 40 ms (accumulate) and 10 ms (split)
feed->encode->relay->decode->sink round trips. ctest --preset dev 25/25.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Talon
2026-06-22 16:45:02 +02:00
parent 50416c33a2
commit a460009a2f
8 changed files with 443 additions and 9 deletions
Download Patch File Download Diff File Expand all files Collapse all files

16
PROGRESS.md
View File

@@ -442,7 +442,21 @@ up instantly. Newest status at the top.
## Recent completed work
All items below are `[x]` done; `ctest --preset dev` 23/23 on Windows after all.
All items below are `[x]` done; `ctest --preset dev` 25/25 on Windows after all.
- **Non-20ms channel frame_ms fix** (2026-06-22): the AudioEngine capture clock is fixed at
48 kHz / 20 ms (960-sample frames), but a channel may set any Opus `frame_ms` (2.5…60 ms,
docs/voice.md §3) and the server enforces it unclamped. The send path handed the engine's
960-sample frame straight to an encoder configured for the channel's window — silently
ignoring `frame_ms > 20` and **breaking `frame_ms < 20` entirely** (receiver sized its decode
buffer too small → `OPUS_BUFFER_TOO_SMALL` → dead audio). Affected the hardware mic AND
`vc_stream_feed_pcm`. Fix: `vc_client::on_capture_frame` now reframes each captured/fed block
to `ls.frame_samples` via a per-`LocalStream` accumulator (pre-sized at announce, no RT-thread
alloc) before `encode_and_send_frame`; the 20 ms case stays a zero-copy fast path. Also pinned
the codec to 48 kHz internally in `opus_params_from_audio_config` (was honoring a non-48k
effective sample_rate against a 48k PCM clock). New ctest `frame_ms_reframe` (40 ms accumulate
+ 10 ms split round trips). Files: `client.{h,cpp}`, `voicecat.h` (feed doc), `docs/voice.md`,
`tests/test_frame_ms_reframe.cpp`, `tests/CMakeLists.txt`.
- **External PCM feed/tap API** (2026-06-20): `vc_stream_feed_pcm` + `vc_set_pcm_sink` shipped.
Promotes `vc_test_inject_capture` (mono-only, TEST-ONLY) to a public, stereo-capable API.