fix(audio): apply receive-side NR to stereo mic streams
The per-listener noise-reduction toggle (vc_set_remote_stream) did nothing on Windows/macOS/iOS. The decode loop gated the RNNoise pass on dec_channels == 1 as a proxy for "this stream is voice" (assuming stereo => screen-share). The stereo-mic capture commit broke that: a stereo mic with send-side NR off transmits stereo Opus, so the receiver decoded two channels and skipped NR entirely. gain/mute have no channel guard, which is why only NR appeared broken. Thread the stream kind through init_recv_stream into RemoteStream::is_voice (set from si.kind() == STREAM_MIC), gate receive NR on is_voice instead of channel count, and fold a stereo voice frame to mono -> denoise -> duplicate back across both channels in place (symmetric with the send-side downmix; RNNoise is mono-only). Screen-audio shares are never denoised. New test test_recv_noise_reduction drives AudioEngine and asserts a stereo voice stream's noise floor collapses with NR on (RMS 1046 -> 0.1) while a screen-audio share stays unchanged. ctest --preset dev green 29/29. Docs: voice.md section 10. Shared-core fix; clients need only a rebuild. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -8,12 +8,13 @@ and what's next* read [`PROGRESS.md`](PROGRESS.md); for *design* read [`docs/`](
|
|||||||
> server-mute, channel CRUD, in-app account management, disconnect/keepalive/reaper. Windows
|
> server-mute, channel CRUD, in-app account management, disconnect/keepalive/reaper. Windows
|
||||||
> WinForms C# client shipped (M4). **macOS AppKit client shipped** — `VoiceCatMac.xcodeproj`
|
> WinForms C# client shipped (M4). **macOS AppKit client shipped** — `VoiceCatMac.xcodeproj`
|
||||||
> at `clients/apple/macOS/`. **iOS SwiftUI client shipped** — `VoiceCatiOS.xcodeproj` at
|
> at `clients/apple/macOS/`. **iOS SwiftUI client shipped** — `VoiceCatiOS.xcodeproj` at
|
||||||
> `clients/apple/iOS/`. `ctest --preset dev` green — 28/28 tests.
|
> `clients/apple/iOS/`. `ctest --preset dev` green — 29/29 tests.
|
||||||
> External PCM feed/tap API (`vc_stream_feed_pcm` + `vc_set_pcm_sink`) shipped.
|
> External PCM feed/tap API (`vc_stream_feed_pcm` + `vc_set_pcm_sink`) shipped.
|
||||||
> **Screen-audio sharing shipped on macOS (ScreenCaptureKit) and iOS (ReplayKit Broadcast
|
> **Screen-audio sharing shipped on macOS (ScreenCaptureKit) and iOS (ReplayKit Broadcast
|
||||||
> Upload Extension → host App Group ring → `vc_stream_feed_pcm`).**
|
> Upload Extension → host App Group ring → `vc_stream_feed_pcm`).**
|
||||||
> **Noise suppression shipped (RNNoise, vendored at `third_party/rnnoise/`)** — both send-side
|
> **Noise suppression shipped (RNNoise, vendored at `third_party/rnnoise/`)** — both send-side
|
||||||
> mic NR (`vc_set_input_noise_reduction`) and per-listener receive NR; client on/off toggles TBD.
|
> mic NR (`vc_set_input_noise_reduction`) and per-listener receive NR; client on/off toggles ship
|
||||||
|
on all three clients (receive NR now denoises stereo mic streams too — fixed 2026-06-23).
|
||||||
> See [`PROGRESS.md`](PROGRESS.md).
|
> See [`PROGRESS.md`](PROGRESS.md).
|
||||||
|
|
||||||
VoiceCat = self-hosted native voice & text chat (TeamSpeak/Mumble-style). Plain TCP (control)
|
VoiceCat = self-hosted native voice & text chat (TeamSpeak/Mumble-style). Plain TCP (control)
|
||||||
|
|||||||
18
PROGRESS.md
18
PROGRESS.md
@@ -26,6 +26,24 @@ up instantly. Newest status at the top.
|
|||||||
send cushion could be reduced or removed. Shared-core change → add a test and re-verify
|
send cushion could be reduced or removed. Shared-core change → add a test and re-verify
|
||||||
desktop↔desktop stays low-latency (steady sender ⇒ ~0 arrival jitter ⇒ no regression).
|
desktop↔desktop stays low-latency (steady sender ⇒ ~0 arrival jitter ⇒ no regression).
|
||||||
|
|
||||||
|
- **Done (2026-06-23):** **Fixed: receive-side noise reduction silently skipped on stereo mic
|
||||||
|
streams (regression from stereo-mic capture below).** The per-listener NR toggle
|
||||||
|
(`vc_set_remote_stream(... noise_reduction)`) did nothing on Windows/macOS/iOS — the UI and
|
||||||
|
the whole C-ABI→core path were correctly wired, but the decode loop gated the RNNoise pass on
|
||||||
|
`dec_channels == 1` (`core/src/audio/audio_engine.cpp`), an old proxy for "this stream is
|
||||||
|
voice" that assumed *stereo ⇒ screen-share*. The stereo-mic commit broke it: a stereo mic with
|
||||||
|
**send-side NR off** transmits stereo Opus, so the receiver decoded `dec_channels == 2` and
|
||||||
|
skipped NR entirely (gain/mute have no channel guard, which is why only NR looked broken).
|
||||||
|
**Fix:** thread the stream *kind* through `init_recv_stream` into `RemoteStream::is_voice`
|
||||||
|
(set from `si.kind() == STREAM_MIC` in `client.cpp`), gate receive NR on `is_voice` instead of
|
||||||
|
channel count, and fold a stereo voice frame to mono → denoise → duplicate back across both
|
||||||
|
channels in place (symmetric with the send-side downmix; RNNoise is mono-only). A stereo voice
|
||||||
|
stream now plays mono while NR is on; a screen-audio share is never touched. New test
|
||||||
|
`tests/test_recv_noise_reduction.cpp` drives `AudioEngine` and asserts a stereo voice stream's
|
||||||
|
noise floor collapses with NR on (RMS 1046 → 0.1) while a screen-audio share stays unchanged
|
||||||
|
(RMS ≈ 1015). Full `ctest --preset dev` green — **29/29**. Docs: voice.md §10. Clients need no
|
||||||
|
change (shared-core fix). Not yet re-verified two-client E2E on real hardware.
|
||||||
|
|
||||||
- **Done (2026-06-23):** **Stereo mic capture on Windows & macOS desktop clients.** Both
|
- **Done (2026-06-23):** **Stereo mic capture on Windows & macOS desktop clients.** Both
|
||||||
desktop mics were hard-mono: `ensure_audio_running()` defaults `capture_channels = 1` and
|
desktop mics were hard-mono: `ensure_audio_running()` defaults `capture_channels = 1` and
|
||||||
neither client ever called `vc_set_capture_channels` (only iOS did). Added a **"Stereo
|
neither client ever called `vc_set_capture_channels` (only iOS did). Added a **"Stereo
|
||||||
|
|||||||
@@ -592,11 +592,12 @@ int32_t AudioEngine::stream_playout_depth_samples(uint32_t ssrc) const {
|
|||||||
|
|
||||||
#ifdef VOICECAT_HAS_OPUS
|
#ifdef VOICECAT_HAS_OPUS
|
||||||
void AudioEngine::init_recv_stream(uint32_t ssrc, const codec::OpusParams& p,
|
void AudioEngine::init_recv_stream(uint32_t ssrc, const codec::OpusParams& p,
|
||||||
uint32_t user_id, uint32_t stream_id) {
|
uint32_t user_id, uint32_t stream_id, bool is_voice) {
|
||||||
std::lock_guard lk(streams_mu_);
|
std::lock_guard lk(streams_mu_);
|
||||||
auto& stream = streams_[ssrc];
|
auto& stream = streams_[ssrc];
|
||||||
stream.user_id = user_id;
|
stream.user_id = user_id;
|
||||||
stream.stream_id = stream_id;
|
stream.stream_id = stream_id;
|
||||||
|
stream.is_voice = is_voice;
|
||||||
stream.fec_enabled_ = p.fec;
|
stream.fec_enabled_ = p.fec;
|
||||||
stream.decoder.init(p);
|
stream.decoder.init(p);
|
||||||
// Ring must be sized for this decoder's actual channel/frame-size — see RemoteStream::ring
|
// Ring must be sized for this decoder's actual channel/frame-size — see RemoteStream::ring
|
||||||
@@ -788,11 +789,25 @@ void AudioEngine::on_playback(int16_t* out, ma_uint32 frames) {
|
|||||||
|
|
||||||
// `n` is samples-per-channel (matches the frame_samples convention used by
|
// `n` is samples-per-channel (matches the frame_samples convention used by
|
||||||
// OpusEncoder::encode elsewhere in the codebase).
|
// OpusEncoder::encode elsewhere in the codebase).
|
||||||
// RNNoise is mono-only; a stereo stream (screen-audio share) is never voice, so skip
|
// Receive-side NR runs only on VOICE (MIC) streams — a screen-audio share is music/
|
||||||
// NR there rather than denoise a garbled deinterleave (docs/voice.md §10).
|
// video, never voice, so it's left untouched (gating on dec_channels would silently
|
||||||
if (stream.recv_ns && dec_channels == 1)
|
// skip a now-stereo mic, see docs/voice.md §10). RNNoise is mono-only, so a stereo mic
|
||||||
stream.recv_ns->process_capture(stream.decode_scratch.data(), n,
|
// is folded to mono in place (symmetric with the send-side downmix), denoised, then
|
||||||
static_cast<int>(params_.sample_rate));
|
// duplicated back across both channels — no allocation on this RT path.
|
||||||
|
if (stream.recv_ns && stream.is_voice) {
|
||||||
|
int16_t* s = stream.decode_scratch.data();
|
||||||
|
if (dec_channels == 2) {
|
||||||
|
for (int i = 0; i < n; ++i) // L/R -> mono, packed into [0..n)
|
||||||
|
s[i] = static_cast<int16_t>(
|
||||||
|
(static_cast<int32_t>(s[2 * i]) + static_cast<int32_t>(s[2 * i + 1])) /
|
||||||
|
2);
|
||||||
|
stream.recv_ns->process_capture(s, n, static_cast<int>(params_.sample_rate));
|
||||||
|
for (int i = n - 1; i >= 0; --i) // mono -> both channels (back-to-front)
|
||||||
|
s[2 * i] = s[2 * i + 1] = s[i];
|
||||||
|
} else if (dec_channels == 1) {
|
||||||
|
stream.recv_ns->process_capture(s, n, static_cast<int>(params_.sample_rate));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// PCM sink: deliver decoded per-stream audio to external consumer (bots,
|
// PCM sink: deliver decoded per-stream audio to external consumer (bots,
|
||||||
// transcription, recording) before it enters the hardware mix. Atomic relaxed-
|
// transcription, recording) before it enters the hardware mix. Atomic relaxed-
|
||||||
|
|||||||
@@ -225,9 +225,11 @@ class AudioEngine {
|
|||||||
#ifdef VOICECAT_HAS_OPUS
|
#ifdef VOICECAT_HAS_OPUS
|
||||||
// Configure the Opus decoder for an incoming ssrc (must be called before
|
// Configure the Opus decoder for an incoming ssrc (must be called before
|
||||||
// push_recv_frame for that ssrc). user_id/stream_id identify the source for the
|
// push_recv_frame for that ssrc). user_id/stream_id identify the source for the
|
||||||
// pcm_sink_ callback. Thread-safe.
|
// pcm_sink_ callback. is_voice marks a MIC stream so the receive-side NR pass knows it may
|
||||||
|
// denoise it (a stereo mic is folded to mono first); screen-audio shares are never voice.
|
||||||
|
// Thread-safe.
|
||||||
void init_recv_stream(uint32_t ssrc, const codec::OpusParams& p,
|
void init_recv_stream(uint32_t ssrc, const codec::OpusParams& p,
|
||||||
uint32_t user_id, uint32_t stream_id);
|
uint32_t user_id, uint32_t stream_id, bool is_voice);
|
||||||
#endif
|
#endif
|
||||||
|
|
||||||
// External PCM tap: callback fired once per decoded Opus frame per remote stream, on the
|
// External PCM tap: callback fired once per decoded Opus frame per remote stream, on the
|
||||||
@@ -475,6 +477,11 @@ class AudioEngine {
|
|||||||
bool noise_reduction_enabled = false;
|
bool noise_reduction_enabled = false;
|
||||||
std::unique_ptr<ApmProcessor> recv_ns;
|
std::unique_ptr<ApmProcessor> recv_ns;
|
||||||
|
|
||||||
|
// True for a MIC stream (voice). The receive-side NR pass only denoises voice — a stereo
|
||||||
|
// mic is folded to mono first (RNNoise is mono-only); a stereo screen-audio share is not
|
||||||
|
// voice and is left untouched. Set in init_recv_stream. (docs/voice.md §10.)
|
||||||
|
bool is_voice = false;
|
||||||
|
|
||||||
// DRED: pre-allocated scratch for loss recovery. dred_state_ is per-stream; see
|
// DRED: pre-allocated scratch for loss recovery. dred_state_ is per-stream; see
|
||||||
// AudioEngine::dred_dec_ (shared). Allocated in init_recv_stream(); freed in remove_stream().
|
// AudioEngine::dred_dec_ (shared). Allocated in init_recv_stream(); freed in remove_stream().
|
||||||
#ifdef VOICECAT_HAS_OPUS
|
#ifdef VOICECAT_HAS_OPUS
|
||||||
|
|||||||
@@ -1229,7 +1229,9 @@ void vc_client::sync_remote_streams(const voicecat::v1::User& user) {
|
|||||||
remote_streams_[ssrc] = {user.id(), si.stream_id()};
|
remote_streams_[ssrc] = {user.id(), si.stream_id()};
|
||||||
|
|
||||||
voicecat::codec::OpusParams p = opus_params_from_audio_config(si.audio());
|
voicecat::codec::OpusParams p = opus_params_from_audio_config(si.audio());
|
||||||
audio_engine_.init_recv_stream(ssrc, p, user.id(), si.stream_id());
|
// Only a MIC stream is voice; receive-side NR denoises voice only (docs/voice.md §10).
|
||||||
|
bool is_voice = si.kind() == voicecat::v1::STREAM_MIC;
|
||||||
|
audio_engine_.init_recv_stream(ssrc, p, user.id(), si.stream_id(), is_voice);
|
||||||
bool muted = self_deafened_.load(std::memory_order_acquire) ||
|
bool muted = self_deafened_.load(std::memory_order_acquire) ||
|
||||||
server_deafened_.load(std::memory_order_acquire);
|
server_deafened_.load(std::memory_order_acquire);
|
||||||
audio_engine_.set_stream_mute(ssrc, muted);
|
audio_engine_.set_stream_mute(ssrc, muted);
|
||||||
|
|||||||
@@ -321,8 +321,12 @@ and every Opus frame size (480/960/1920/2880) is a multiple of 480, so frames ar
|
|||||||
whole 480-sample chunks with no resampling. Because it's mono-only:
|
whole 480-sample chunks with no resampling. Because it's mono-only:
|
||||||
- **Send-side:** a stereo mic is downmixed to mono **only when NR is enabled** — with NR off a
|
- **Send-side:** a stereo mic is downmixed to mono **only when NR is enabled** — with NR off a
|
||||||
stereo mic keeps full stereo (we never collapse mic quality unless asked).
|
stereo mic keeps full stereo (we never collapse mic quality unless asked).
|
||||||
- **Receive-side:** NR is skipped on stereo streams (a stereo stream is a screen-audio share,
|
- **Receive-side:** NR applies to **voice (MIC) streams only**, gated on the stream *kind* — not
|
||||||
not voice).
|
on its channel count, since a stereo mic with send-side NR off now arrives as stereo voice.
|
||||||
|
When enabled on such a stream the decoded stereo frame is folded to mono, denoised, and
|
||||||
|
duplicated back across both channels (symmetric with the send-side downmix), so that stream
|
||||||
|
plays as mono while NR is on. A **screen-audio share is never voice and is left untouched** —
|
||||||
|
denoising music/video with a speech denoiser would mangle it.
|
||||||
|
|
||||||
Implementation: a per-`ssrc` NS instance (`RemoteStream::recv_ns`) on the receive path,
|
Implementation: a per-`ssrc` NS instance (`RemoteStream::recv_ns`) on the receive path,
|
||||||
instantiated lazily only for streams the listener has flagged; the send-side instance
|
instantiated lazily only for streams the listener has flagged; the send-side instance
|
||||||
|
|||||||
@@ -237,4 +237,14 @@ if(VOICECAT_USE_VCPKG_DEPS)
|
|||||||
target_compile_features(test_noise_suppression PRIVATE cxx_std_20)
|
target_compile_features(test_noise_suppression PRIVATE cxx_std_20)
|
||||||
target_include_directories(test_noise_suppression PRIVATE ${VC_TEST_INTERNAL_INCLUDES})
|
target_include_directories(test_noise_suppression PRIVATE ${VC_TEST_INTERNAL_INCLUDES})
|
||||||
add_test(NAME noise_suppression COMMAND test_noise_suppression)
|
add_test(NAME noise_suppression COMMAND test_noise_suppression)
|
||||||
|
|
||||||
|
# Receive-side NR through the decode loop: a stereo VOICE stream is actually denoised
|
||||||
|
# (regression for the stereo-mic bypass), and a screen-audio share is left untouched.
|
||||||
|
# White-box AudioEngine test; paced realtime feed, so it needs a generous timeout.
|
||||||
|
add_executable(test_recv_noise_reduction test_recv_noise_reduction.cpp)
|
||||||
|
target_link_libraries(test_recv_noise_reduction PRIVATE voicecat::voicecat)
|
||||||
|
target_compile_features(test_recv_noise_reduction PRIVATE cxx_std_20)
|
||||||
|
target_include_directories(test_recv_noise_reduction PRIVATE ${VC_TEST_INTERNAL_INCLUDES})
|
||||||
|
add_test(NAME recv_noise_reduction COMMAND test_recv_noise_reduction)
|
||||||
|
set_tests_properties(recv_noise_reduction PROPERTIES TIMEOUT 90)
|
||||||
endif()
|
endif()
|
||||||
|
|||||||
@@ -91,7 +91,7 @@ int main() {
|
|||||||
CHECK(opus_len > 0);
|
CHECK(opus_len > 0);
|
||||||
|
|
||||||
const uint32_t ssrc = 1;
|
const uint32_t ssrc = 1;
|
||||||
engine.init_recv_stream(ssrc, op, /*user_id=*/7, /*stream_id=*/3);
|
engine.init_recv_stream(ssrc, op, /*user_id=*/7, /*stream_id=*/3, /*is_voice=*/false);
|
||||||
|
|
||||||
// ── Phase 1: feed ~600ms of real frames; the timer must decode + mix them. ──────────
|
// ── Phase 1: feed ~600ms of real frames; the timer must decode + mix them. ──────────
|
||||||
uint32_t ts = 0;
|
uint32_t ts = 0;
|
||||||
|
|||||||
@@ -67,7 +67,7 @@ int main() {
|
|||||||
CHECK(opus_len > 0);
|
CHECK(opus_len > 0);
|
||||||
|
|
||||||
const uint32_t ssrc = 1;
|
const uint32_t ssrc = 1;
|
||||||
engine.init_recv_stream(ssrc, op, /*user_id=*/0, /*stream_id=*/0);
|
engine.init_recv_stream(ssrc, op, /*user_id=*/0, /*stream_id=*/0, /*is_voice=*/false);
|
||||||
|
|
||||||
const uint32_t pb_frames = 480; // 10 ms hardware period
|
const uint32_t pb_frames = 480; // 10 ms hardware period
|
||||||
const int out_n = static_cast<int>(pb_frames) * 2; // stereo interleaved
|
const int out_n = static_cast<int>(pb_frames) * 2; // stereo interleaved
|
||||||
|
|||||||
@@ -69,7 +69,7 @@ int main() {
|
|||||||
CHECK(opus_len > 0);
|
CHECK(opus_len > 0);
|
||||||
|
|
||||||
const uint32_t ssrc = 1;
|
const uint32_t ssrc = 1;
|
||||||
engine.init_recv_stream(ssrc, op, /*user_id=*/0, /*stream_id=*/0);
|
engine.init_recv_stream(ssrc, op, /*user_id=*/0, /*stream_id=*/0, /*is_voice=*/false);
|
||||||
|
|
||||||
// Push one real frame to seed the decoder.
|
// Push one real frame to seed the decoder.
|
||||||
voicecat::audio::JitterBuffer::Frame f;
|
voicecat::audio::JitterBuffer::Frame f;
|
||||||
|
|||||||
180
tests/test_recv_noise_reduction.cpp
Normal file
180
tests/test_recv_noise_reduction.cpp
Normal file
@@ -0,0 +1,180 @@
|
|||||||
|
/*
|
||||||
|
* test_recv_noise_reduction — receive-side NR actually denoises a STEREO voice stream, and
|
||||||
|
* never touches a screen-audio share. Regression for the bug where a stereo mic (shipped in
|
||||||
|
* f72219d) silently bypassed listener-side NR because the decode loop gated on
|
||||||
|
* `dec_channels == 1` as a proxy for "is voice" (docs/voice.md §10).
|
||||||
|
*
|
||||||
|
* White-box: drive AudioEngine in external-playback mode (the mixer-timer thread decodes+mixes
|
||||||
|
* with no hardware device, same as test_external_playback) and tap the per-stream pcm_sink, which
|
||||||
|
* fires AFTER the receive-side NR pass. We feed deterministic stereo white noise (paced at the
|
||||||
|
* 20 ms engine cadence so the jitter buffer's catch-up never drops the backlog) and compare the
|
||||||
|
* played-out noise floor across three runs:
|
||||||
|
* • voice stream, NR off → baseline (decode only)
|
||||||
|
* • voice stream, NR on → RNNoise folds stereo→mono and collapses the noise floor
|
||||||
|
* • screen-audio (is_voice=false), NR on → NR is skipped; floor ≈ baseline
|
||||||
|
*
|
||||||
|
* Registered only under VOICECAT_USE_VCPKG_DEPS, where VOICECAT_HAS_NS is defined (a passthrough
|
||||||
|
* build would correctly fail the reduction assertion).
|
||||||
|
*/
|
||||||
|
#include <atomic>
|
||||||
|
#include <chrono>
|
||||||
|
#include <cmath>
|
||||||
|
#include <cstdint>
|
||||||
|
#include <cstdio>
|
||||||
|
#include <thread>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
#if defined(VOICECAT_HAS_AUDIO) && defined(VOICECAT_HAS_OPUS)
|
||||||
|
|
||||||
|
#include "audio/audio_engine.h"
|
||||||
|
#include "codec/opus_codec.h"
|
||||||
|
|
||||||
|
static int g_failures = 0;
|
||||||
|
#define CHECK(cond) \
|
||||||
|
do { \
|
||||||
|
if (!(cond)) { \
|
||||||
|
std::printf("FAIL [%s:%d]: %s\n", __FILE__, __LINE__, #cond); \
|
||||||
|
++g_failures; \
|
||||||
|
} \
|
||||||
|
} while (0)
|
||||||
|
|
||||||
|
namespace vca = voicecat::audio;
|
||||||
|
namespace vcc = voicecat::codec;
|
||||||
|
|
||||||
|
// pcm_sink accumulator (written on the mixer-timer thread, read by main after stop()).
|
||||||
|
struct Sink {
|
||||||
|
std::atomic<int> frames{0};
|
||||||
|
std::atomic<long long> sumsq{0};
|
||||||
|
std::atomic<long long> count{0};
|
||||||
|
std::atomic<uint32_t> last_ch{0};
|
||||||
|
int warmup = 0; // skip RNNoise's recurrent-state settle window
|
||||||
|
};
|
||||||
|
|
||||||
|
static void sink_cb(void* user, uint32_t, uint32_t, const int16_t* pcm, size_t spc, uint32_t ch,
|
||||||
|
uint32_t) {
|
||||||
|
auto* s = static_cast<Sink*>(user);
|
||||||
|
int f = s->frames.fetch_add(1, std::memory_order_relaxed);
|
||||||
|
s->last_ch.store(ch, std::memory_order_relaxed);
|
||||||
|
if (f < s->warmup) return;
|
||||||
|
const size_t total = spc * ch;
|
||||||
|
long long sq = 0;
|
||||||
|
for (size_t i = 0; i < total; ++i) {
|
||||||
|
long long v = pcm[i];
|
||||||
|
sq += v * v;
|
||||||
|
}
|
||||||
|
s->sumsq.fetch_add(sq, std::memory_order_relaxed);
|
||||||
|
s->count.fetch_add(static_cast<long long>(total), std::memory_order_relaxed);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Run one playout scenario and return the RMS of the played-out PCM, or -1 if nothing played.
|
||||||
|
static double run_scenario(bool is_voice, bool nr_on,
|
||||||
|
const std::vector<std::vector<uint8_t>>& frames, const vcc::OpusParams& op,
|
||||||
|
int frame_samples) {
|
||||||
|
vca::AudioEngine engine;
|
||||||
|
engine.set_external_playback(true);
|
||||||
|
Sink sink;
|
||||||
|
sink.warmup = 20;
|
||||||
|
engine.set_pcm_sink(&sink_cb, &sink);
|
||||||
|
|
||||||
|
vca::AudioParams p;
|
||||||
|
p.sample_rate = 48000;
|
||||||
|
p.capture_channels = 1;
|
||||||
|
p.playback_channels = 2;
|
||||||
|
p.frame_ms = 20;
|
||||||
|
if (!engine.start(p)) return -1;
|
||||||
|
|
||||||
|
const uint32_t ssrc = 1;
|
||||||
|
engine.init_recv_stream(ssrc, op, /*user_id=*/7, /*stream_id=*/3, is_voice);
|
||||||
|
if (nr_on) engine.set_stream_noise_reduction(ssrc, true);
|
||||||
|
|
||||||
|
uint32_t ts = 0;
|
||||||
|
for (size_t i = 0; i < frames.size(); ++i) {
|
||||||
|
vca::JitterBuffer::Frame f;
|
||||||
|
f.seq = static_cast<uint64_t>(i);
|
||||||
|
f.timestamp = ts;
|
||||||
|
f.fec_present = false;
|
||||||
|
f.payload = frames[i];
|
||||||
|
engine.push_recv_frame(ssrc, std::move(f));
|
||||||
|
ts += static_cast<uint32_t>(frame_samples);
|
||||||
|
// Pace at the engine cadence so production ≈ consumption: the jitter buffer never builds a
|
||||||
|
// backlog big enough to trigger catch-up frame-dropping, so every frame is decoded.
|
||||||
|
std::this_thread::sleep_for(std::chrono::milliseconds(20));
|
||||||
|
}
|
||||||
|
std::this_thread::sleep_for(std::chrono::milliseconds(120)); // drain
|
||||||
|
engine.stop();
|
||||||
|
|
||||||
|
CHECK(sink.last_ch.load(std::memory_order_relaxed) == 2); // stereo throughout
|
||||||
|
long long c = sink.count.load(std::memory_order_relaxed);
|
||||||
|
if (c == 0) return -1;
|
||||||
|
return std::sqrt(static_cast<double>(sink.sumsq.load(std::memory_order_relaxed)) /
|
||||||
|
static_cast<double>(c));
|
||||||
|
}
|
||||||
|
|
||||||
|
int main() {
|
||||||
|
// Stereo Opus params (a stereo mic with send-side NR off arrives exactly like this).
|
||||||
|
vcc::OpusParams op;
|
||||||
|
op.sample_rate = 48000;
|
||||||
|
op.frame_ms = 20;
|
||||||
|
op.stereo = true;
|
||||||
|
int frame_samples = vcc::opus_frame_samples(op); // 960 per channel
|
||||||
|
|
||||||
|
// Encode a run of deterministic interleaved-stereo white noise (L and R independent so the
|
||||||
|
// bitstream is genuinely stereo, not L==R).
|
||||||
|
uint32_t rng = 0xC0FFEEu;
|
||||||
|
auto next_noise = [&]() -> int16_t {
|
||||||
|
rng ^= rng << 13;
|
||||||
|
rng ^= rng >> 17;
|
||||||
|
rng ^= rng << 5;
|
||||||
|
return static_cast<int16_t>((static_cast<int32_t>(rng % 6001)) - 3000); // ~[-3000,3000]
|
||||||
|
};
|
||||||
|
|
||||||
|
vcc::OpusEncoder enc;
|
||||||
|
CHECK(enc.init(op));
|
||||||
|
const int kFrames = 80; // ~1.6 s; warmup 20 leaves ~60 measured
|
||||||
|
std::vector<std::vector<uint8_t>> frames;
|
||||||
|
frames.reserve(kFrames);
|
||||||
|
std::vector<int16_t> interleaved(static_cast<size_t>(frame_samples) * 2);
|
||||||
|
uint8_t opus_buf[1500];
|
||||||
|
for (int f = 0; f < kFrames; ++f) {
|
||||||
|
for (int i = 0; i < frame_samples * 2; ++i) interleaved[i] = next_noise();
|
||||||
|
int len = enc.encode(interleaved.data(), frame_samples, opus_buf, sizeof(opus_buf));
|
||||||
|
CHECK(len > 0);
|
||||||
|
frames.emplace_back(opus_buf, opus_buf + len);
|
||||||
|
}
|
||||||
|
enc.destroy();
|
||||||
|
|
||||||
|
double voice_off = run_scenario(/*is_voice=*/true, /*nr_on=*/false, frames, op, frame_samples);
|
||||||
|
double voice_on = run_scenario(/*is_voice=*/true, /*nr_on=*/true, frames, op, frame_samples);
|
||||||
|
double screen_on = run_scenario(/*is_voice=*/false, /*nr_on=*/true, frames, op, frame_samples);
|
||||||
|
|
||||||
|
std::printf("recv_nr: voice_off_rms=%.1f voice_on_rms=%.1f screen_on_rms=%.1f\n", voice_off,
|
||||||
|
voice_on, screen_on);
|
||||||
|
|
||||||
|
CHECK(voice_off > 0.0);
|
||||||
|
CHECK(voice_on > 0.0);
|
||||||
|
CHECK(screen_on > 0.0);
|
||||||
|
|
||||||
|
// The fix: enabling NR on the stereo VOICE stream must collapse the noise floor. RNNoise drops
|
||||||
|
// pure noise ~99%; require a clear, unambiguous reduction (the pre-fix bug left it unchanged).
|
||||||
|
CHECK(voice_on < voice_off * 0.6);
|
||||||
|
|
||||||
|
// A screen-audio share is not voice: NR must be a no-op there, so its floor stays ≈ the
|
||||||
|
// undenoised baseline (same decode path, no RNNoise). Loose band absorbs run-to-run timing.
|
||||||
|
CHECK(screen_on > voice_off * 0.7);
|
||||||
|
|
||||||
|
if (g_failures == 0) {
|
||||||
|
std::printf("recv_noise_reduction: OK\n");
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
std::printf("recv_noise_reduction: %d failure(s)\n", g_failures);
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
#else
|
||||||
|
|
||||||
|
int main() {
|
||||||
|
std::printf("recv_noise_reduction: SKIP (VOICECAT_HAS_AUDIO or VOICECAT_HAS_OPUS not defined)\n");
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
#endif
|
||||||
@@ -205,7 +205,8 @@ static void test_stereo_mix() {
|
|||||||
int opus_len = enc.encode(interleaved.data(), frame_samples, opus_buf, sizeof(opus_buf));
|
int opus_len = enc.encode(interleaved.data(), frame_samples, opus_buf, sizeof(opus_buf));
|
||||||
CHECK(opus_len > 0);
|
CHECK(opus_len > 0);
|
||||||
|
|
||||||
engine.init_recv_stream(/*ssrc=*/1, stereo_params, /*user_id=*/0, /*stream_id=*/0);
|
engine.init_recv_stream(/*ssrc=*/1, stereo_params, /*user_id=*/0, /*stream_id=*/0,
|
||||||
|
/*is_voice=*/false);
|
||||||
|
|
||||||
voicecat::audio::JitterBuffer::Frame f;
|
voicecat::audio::JitterBuffer::Frame f;
|
||||||
f.seq = 0;
|
f.seq = 0;
|
||||||
@@ -291,7 +292,8 @@ static void test_loopback_stereo_capture() {
|
|||||||
|
|
||||||
// Decode + mix — same recv path as test_stereo_mix. A real stereo bitstream should
|
// Decode + mix — same recv path as test_stereo_mix. A real stereo bitstream should
|
||||||
// survive with L != R; a mono-downmixed-then-upmixed bitstream would have L == R.
|
// survive with L != R; a mono-downmixed-then-upmixed bitstream would have L == R.
|
||||||
engine.init_recv_stream(/*ssrc=*/3, stereo_params, /*user_id=*/0, /*stream_id=*/0);
|
engine.init_recv_stream(/*ssrc=*/3, stereo_params, /*user_id=*/0, /*stream_id=*/0,
|
||||||
|
/*is_voice=*/false);
|
||||||
voicecat::audio::JitterBuffer::Frame f;
|
voicecat::audio::JitterBuffer::Frame f;
|
||||||
f.seq = 0;
|
f.seq = 0;
|
||||||
f.timestamp = 0;
|
f.timestamp = 0;
|
||||||
@@ -345,7 +347,8 @@ static void test_playout_resync() {
|
|||||||
int opus_len = enc.encode(sine.data(), frame_samples, opus_buf, sizeof(opus_buf));
|
int opus_len = enc.encode(sine.data(), frame_samples, opus_buf, sizeof(opus_buf));
|
||||||
CHECK(opus_len > 0);
|
CHECK(opus_len > 0);
|
||||||
|
|
||||||
engine.init_recv_stream(/*ssrc=*/2, mono_params, /*user_id=*/0, /*stream_id=*/0);
|
engine.init_recv_stream(/*ssrc=*/2, mono_params, /*user_id=*/0, /*stream_id=*/0,
|
||||||
|
/*is_voice=*/false);
|
||||||
|
|
||||||
std::vector<int16_t> out(static_cast<size_t>(frame_samples) * 2, 0);
|
std::vector<int16_t> out(static_cast<size_t>(frame_samples) * 2, 0);
|
||||||
|
|
||||||
@@ -639,7 +642,8 @@ static void test_stereo_mic_capture() {
|
|||||||
|
|
||||||
// Decode + mix — same recv path as test_stereo_mix. A real stereo bitstream should
|
// Decode + mix — same recv path as test_stereo_mix. A real stereo bitstream should
|
||||||
// survive with L != R; a mono-downmixed-then-upmixed bitstream would have L == R.
|
// survive with L != R; a mono-downmixed-then-upmixed bitstream would have L == R.
|
||||||
engine.init_recv_stream(/*ssrc=*/5, stereo_params, /*user_id=*/0, /*stream_id=*/0);
|
engine.init_recv_stream(/*ssrc=*/5, stereo_params, /*user_id=*/0, /*stream_id=*/0,
|
||||||
|
/*is_voice=*/false);
|
||||||
voicecat::audio::JitterBuffer::Frame f;
|
voicecat::audio::JitterBuffer::Frame f;
|
||||||
f.seq = 0;
|
f.seq = 0;
|
||||||
f.timestamp = 0;
|
f.timestamp = 0;
|
||||||
|
|||||||
Reference in New Issue
Block a user