Talon/voice-cat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(audio): stereo mic capture on Windows & macOS desktop clients
Some checks failed
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Details
Build Linux Binaries / linux/arm64 (push) Has been cancelled
Details

Browse Source 
Both desktop mics were hard-mono: the core defaults capture_channels=1 and
neither client ever called vc_set_capture_channels (only iOS did). Add a
persisted "Stereo microphone" toggle to each client's Audio settings, applied
when the mic stream starts and live via vc_set_capture_channels + vc_audio_restart.
Expose both ABI calls in the Windows interop; the macOS wrapper already had them.

Core fix: encode_and_send_frame now folds a stereo mic frame to mono on a mono
channel - previously the channels==2 branch encoded interleaved L/R directly even
on a mono channel, feeding a mono opus_encode 2x its samples (wrong pitch/garbage).
Real stereo still only reaches the wire on a stereo channel; on a mono channel the
mic is cleanly downmixed.

Test: test_stereo_mic_mono_channel. ctest --preset dev green (28/28). Docs: voice.md.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Talon
2026-06-23 20:48:26 +02:00
parent b14cf2a4e8
commit f72219ddf3
11 changed files with 210 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files

23
core/src/core/client.cpp
View File

@@ -1106,15 +1106,26 @@ void vc_client::encode_and_send_frame(LocalStream& ls, const int16_t* pcm, int s
int channels, int fd) {
uint8_t opus_buf[1500];
int opus_len;
if (channels == 2) {
// Real interleaved stereo PCM (SCREEN_AUDIO loopback on a stereo channel) — encode
// directly, no upmix. `samples` is samples-per-channel, as OpusEncoder::encode expects.
if (channels == 2 && ls.effective_params.stereo) {
// Real interleaved stereo PCM (a stereo mic via vc_set_capture_channels, or SCREEN_AUDIO
// loopback) on a channel configured for stereo — encode directly, true L/R, no fold.
// `samples` is samples-per-channel, as OpusEncoder::encode expects.
opus_len = ls.encoder.encode(pcm, samples, opus_buf, sizeof(opus_buf));
} else if (channels == 2) {
// Stereo capture (stereo mic / line-in) on a MONO channel: the encoder is mono, so fold
// L/R to mono first. Feeding interleaved pairs straight to a mono opus_encode would make
// it read 2× the samples it should (wrong pitch / garbage). upmix_scratch is pre-sized at
// announce and easily holds `samples` mono values. This keeps a stereo-mic toggle safe on
// every channel: real L/R when the channel is stereo, a clean downmix when it isn't.
int16_t* mono = ls.upmix_scratch.data();
for (int i = 0; i < samples; ++i)
mono[i] = static_cast<int16_t>(
(static_cast<int32_t>(pcm[2 * i]) + static_cast<int32_t>(pcm[2 * i + 1])) / 2);
opus_len = ls.encoder.encode(mono, samples, opus_buf, sizeof(opus_buf));
} else if (ls.effective_params.stereo) {
// Mono capture (mic, or loopback on a mono channel, or test injection) on a channel
// Mono capture (mono mic, or loopback on a mono channel, or test injection) on a channel
// configured for stereo — upmix L=R so the stream is still a spec-correct stereo Opus
// bitstream. (Mic stays mono in v1 — no stereo capture device — but a stereo channel
// requires a stereo bitstream, hence the upmix.) upmix_scratch is pre-sized at announce.
// bitstream (a stereo channel requires a stereo bitstream). upmix_scratch is pre-sized at announce.
int16_t* st = ls.upmix_scratch.data();
for (int i = 0; i < samples; ++i) {
st[i * 2] = pcm[i];