Talon/voice-cat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(audio): apply receive-side NR to stereo mic streams
Some checks failed
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Details
Build Linux Binaries / linux/arm64 (push) Has been cancelled
Details

Browse Source 
The per-listener noise-reduction toggle (vc_set_remote_stream) did nothing
on Windows/macOS/iOS. The decode loop gated the RNNoise pass on
dec_channels == 1 as a proxy for "this stream is voice" (assuming
stereo => screen-share). The stereo-mic capture commit broke that: a stereo
mic with send-side NR off transmits stereo Opus, so the receiver decoded
two channels and skipped NR entirely. gain/mute have no channel guard, which
is why only NR appeared broken.

Thread the stream kind through init_recv_stream into RemoteStream::is_voice
(set from si.kind() == STREAM_MIC), gate receive NR on is_voice instead of
channel count, and fold a stereo voice frame to mono -> denoise -> duplicate
back across both channels in place (symmetric with the send-side downmix;
RNNoise is mono-only). Screen-audio shares are never denoised.

New test test_recv_noise_reduction drives AudioEngine and asserts a stereo
voice stream's noise floor collapses with NR on (RMS 1046 -> 0.1) while a
screen-audio share stays unchanged. ctest --preset dev green 29/29.
Docs: voice.md section 10. Shared-core fix; clients need only a rebuild.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Talon
2026-06-23 21:11:03 +02:00
parent f72219ddf3
commit b44a200b95
12 changed files with 261 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files

12
tests/test_vad_ptt_devices.cpp
View File

@@ -205,7 +205,8 @@ static void test_stereo_mix() {
int opus_len = enc.encode(interleaved.data(), frame_samples, opus_buf, sizeof(opus_buf));
CHECK(opus_len > 0);
engine.init_recv_stream(/*ssrc=*/1, stereo_params, /*user_id=*/0, /*stream_id=*/0);
engine.init_recv_stream(/*ssrc=*/1, stereo_params, /*user_id=*/0, /*stream_id=*/0,
/*is_voice=*/false);
voicecat::audio::JitterBuffer::Frame f;
f.seq = 0;
@@ -291,7 +292,8 @@ static void test_loopback_stereo_capture() {
// Decode + mix — same recv path as test_stereo_mix. A real stereo bitstream should
// survive with L != R; a mono-downmixed-then-upmixed bitstream would have L == R.
engine.init_recv_stream(/*ssrc=*/3, stereo_params, /*user_id=*/0, /*stream_id=*/0);
engine.init_recv_stream(/*ssrc=*/3, stereo_params, /*user_id=*/0, /*stream_id=*/0,
/*is_voice=*/false);
voicecat::audio::JitterBuffer::Frame f;
f.seq = 0;
f.timestamp = 0;
@@ -345,7 +347,8 @@ static void test_playout_resync() {
int opus_len = enc.encode(sine.data(), frame_samples, opus_buf, sizeof(opus_buf));
CHECK(opus_len > 0);
engine.init_recv_stream(/*ssrc=*/2, mono_params, /*user_id=*/0, /*stream_id=*/0);
engine.init_recv_stream(/*ssrc=*/2, mono_params, /*user_id=*/0, /*stream_id=*/0,
/*is_voice=*/false);
std::vector<int16_t> out(static_cast<size_t>(frame_samples) * 2, 0);
@@ -639,7 +642,8 @@ static void test_stereo_mic_capture() {
// Decode + mix — same recv path as test_stereo_mix. A real stereo bitstream should
// survive with L != R; a mono-downmixed-then-upmixed bitstream would have L == R.
engine.init_recv_stream(/*ssrc=*/5, stereo_params, /*user_id=*/0, /*stream_id=*/0);
engine.init_recv_stream(/*ssrc=*/5, stereo_params, /*user_id=*/0, /*stream_id=*/0,
/*is_voice=*/false);
voicecat::audio::JitterBuffer::Frame f;
f.seq = 0;
f.timestamp = 0;