Talon/voice-cat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix(audio): apply receive-side NR to stereo mic streams
Some checks failed
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Details
Build Linux Binaries / linux/arm64 (push) Has been cancelled
Details

Browse Source 
The per-listener noise-reduction toggle (vc_set_remote_stream) did nothing
on Windows/macOS/iOS. The decode loop gated the RNNoise pass on
dec_channels == 1 as a proxy for "this stream is voice" (assuming
stereo => screen-share). The stereo-mic capture commit broke that: a stereo
mic with send-side NR off transmits stereo Opus, so the receiver decoded
two channels and skipped NR entirely. gain/mute have no channel guard, which
is why only NR appeared broken.

Thread the stream kind through init_recv_stream into RemoteStream::is_voice
(set from si.kind() == STREAM_MIC), gate receive NR on is_voice instead of
channel count, and fold a stereo voice frame to mono -> denoise -> duplicate
back across both channels in place (symmetric with the send-side downmix;
RNNoise is mono-only). Screen-audio shares are never denoised.

New test test_recv_noise_reduction drives AudioEngine and asserts a stereo
voice stream's noise floor collapses with NR on (RMS 1046 -> 0.1) while a
screen-audio share stays unchanged. ctest --preset dev green 29/29.
Docs: voice.md section 10. Shared-core fix; clients need only a rebuild.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Talon
2026-06-23 21:11:03 +02:00
parent f72219ddf3
commit b44a200b95
12 changed files with 261 additions and 20 deletions
Download Patch File Download Diff File Expand all files Collapse all files

18
PROGRESS.md
View File

@@ -26,6 +26,24 @@ up instantly. Newest status at the top.
send cushion could be reduced or removed. Shared-core change → add a test and re-verify
desktop↔desktop stays low-latency (steady sender ⇒ ~0 arrival jitter ⇒ no regression).
- **Done (2026-06-23):** **Fixed: receive-side noise reduction silently skipped on stereo mic
streams (regression from stereo-mic capture below).** The per-listener NR toggle
(`vc_set_remote_stream(... noise_reduction)`) did nothing on Windows/macOS/iOS — the UI and
the whole C-ABI→core path were correctly wired, but the decode loop gated the RNNoise pass on
`dec_channels == 1` (`core/src/audio/audio_engine.cpp`), an old proxy for "this stream is
voice" that assumed *stereo ⇒ screen-share*. The stereo-mic commit broke it: a stereo mic with
**send-side NR off** transmits stereo Opus, so the receiver decoded `dec_channels == 2` and
skipped NR entirely (gain/mute have no channel guard, which is why only NR looked broken).
**Fix:** thread the stream *kind* through `init_recv_stream` into `RemoteStream::is_voice`
(set from `si.kind() == STREAM_MIC` in `client.cpp`), gate receive NR on `is_voice` instead of
channel count, and fold a stereo voice frame to mono → denoise → duplicate back across both
channels in place (symmetric with the send-side downmix; RNNoise is mono-only). A stereo voice
stream now plays mono while NR is on; a screen-audio share is never touched. New test
`tests/test_recv_noise_reduction.cpp` drives `AudioEngine` and asserts a stereo voice stream's
noise floor collapses with NR on (RMS 1046 → 0.1) while a screen-audio share stays unchanged
(RMS ≈ 1015). Full `ctest --preset dev` green — **29/29**. Docs: voice.md §10. Clients need no
change (shared-core fix). Not yet re-verified two-client E2E on real hardware.
- **Done (2026-06-23):** **Stereo mic capture on Windows & macOS desktop clients.** Both
desktop mics were hard-mono: `ensure_audio_running()` defaults `capture_channels = 1` and
neither client ever called `vc_set_capture_channels` (only iOS did). Added a **"Stereo