feat(audio): real noise suppression via vendored RNNoise (send + receive)
The two-sided NR plumbing (RemoteStream::recv_ns + the per-listener vc_set_remote_stream noise_reduction toggle) was wired but inert: ApmProcessor::create() returned a no-op passthrough, because the originally-planned webrtc-audio-processing has no working Windows/macOS build. Drop in RNNoise as the real backend behind the same ApmProcessor interface, lighting up both NR paths. - Vendor RNNoise (BSD-3 + CC0) at third_party/rnnoise/ — the vcpkg port is !windows !arm, so it can't cover our primary targets. Shrunk int8 model (78MB -> 11.7MB via upstream scripts/shrink_model.sh), built as a standalone C static lib with no RTCD (portable scalar path on x86, auto-NEON on arm64) under -DDISABLE_DEBUG_FLOAT. Model is baked in (rnnoise_create(NULL)); no runtime file. - New RnnoiseProcessor (core/src/audio/apm_processor.cpp) selected by ApmProcessor::create() when VOICECAT_HAS_NS. Mono/48kHz/480-sample; our clock is fixed 48kHz and Opus frame sizes are multiples of 480, so no resampling. RT-safe: allocates at construction, lock-free in the capture/playback callbacks. - Receive-side: lit up via the factory; gated to mono streams (a stereo stream is a screen-audio share, not voice). - Send-side (new): vc_set_input_noise_reduction(client, enable) ABI + vc_client::mic_ns_, run before input gain/VAD in on_capture_frame. A stereo mic is downmixed to mono ONLY when NR is on — with NR off a stereo mic keeps full stereo (never collapse mic quality unasked). - Enable C as a project language for the vendored lib. - New noise_suppression test: white noise through ApmProcessor::create() drops ~99.9% RMS. ctest --preset dev green, 28/28. windows-client DLL builds clean with vc_set_input_noise_reduction exported, system-only deps. - Docs synced: voice.md §10, tech-stack.md §1/§5, third_party/README.md, vcpkg.json note, PROGRESS.md, CLAUDE.md. Client on/off UI toggles (Windows/macOS/iOS) are the remaining follow-up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
100
tests/test_noise_suppression.cpp
Normal file
100
tests/test_noise_suppression.cpp
Normal file
@@ -0,0 +1,100 @@
|
||||
/*
|
||||
* test_noise_suppression — the RNNoise backend behind ApmProcessor actually denoises.
|
||||
*
|
||||
* This is the behavior exit-criterion for the noise-suppression feature (docs/voice.md §10-11):
|
||||
* a real DSP backend, not the old inert passthrough. ApmProcessor::create() returns the RNNoise
|
||||
* processor when the core is built with VOICECAT_HAS_NS (the dev/release presets). We feed it
|
||||
* mono 48 kHz white noise in 20 ms (960-sample) frames — exercising the internal 480-sample
|
||||
* chunking — and assert the output noise floor collapses while values stay finite/in-range.
|
||||
*
|
||||
* Registered only under VOICECAT_USE_VCPKG_DEPS, where VOICECAT_HAS_NS is defined, so a large
|
||||
* reduction is expected; a passthrough build would (correctly) fail this test.
|
||||
*/
|
||||
#include <cmath>
|
||||
#include <cstdint>
|
||||
#include <cstdio>
|
||||
#include <vector>
|
||||
|
||||
#include "audio/apm_processor.h"
|
||||
|
||||
namespace vca = voicecat::audio;
|
||||
|
||||
static int g_failures = 0;
|
||||
|
||||
#define CHECK(cond) \
|
||||
do { \
|
||||
if (!(cond)) { \
|
||||
std::printf("FAIL [%s:%d]: %s\n", __FILE__, __LINE__, #cond); \
|
||||
++g_failures; \
|
||||
} \
|
||||
} while (0)
|
||||
|
||||
static constexpr int kFrameSamples = 960; // 20 ms @ 48 kHz (two RNNoise 480-sample frames)
|
||||
|
||||
int main() {
|
||||
auto ns = vca::ApmProcessor::create();
|
||||
CHECK(ns != nullptr);
|
||||
if (!ns) return 1;
|
||||
|
||||
// Deterministic white noise (xorshift) at ~int16/10 amplitude, processed frame by frame.
|
||||
uint32_t rng = 0x12345678u;
|
||||
auto next_noise = [&]() -> int16_t {
|
||||
rng ^= rng << 13;
|
||||
rng ^= rng >> 17;
|
||||
rng ^= rng << 5;
|
||||
// map to roughly [-3000, 3000]
|
||||
return static_cast<int16_t>((static_cast<int32_t>(rng % 6001)) - 3000);
|
||||
};
|
||||
|
||||
const int kFrames = 200;
|
||||
const int kWarmup = 60; // let RNNoise's recurrent state settle before measuring
|
||||
double in_sumsq = 0.0, out_sumsq = 0.0;
|
||||
long measured = 0;
|
||||
std::vector<int16_t> frame(kFrameSamples);
|
||||
|
||||
for (int f = 0; f < kFrames; ++f) {
|
||||
double frame_in_sq = 0.0;
|
||||
for (int i = 0; i < kFrameSamples; ++i) {
|
||||
frame[i] = next_noise();
|
||||
frame_in_sq += static_cast<double>(frame[i]) * frame[i];
|
||||
}
|
||||
bool gate = ns->process_capture(frame.data(), kFrameSamples, 48000);
|
||||
CHECK(gate); // NS never gates — always passes the frame on
|
||||
|
||||
if (f >= kWarmup) {
|
||||
in_sumsq += frame_in_sq;
|
||||
for (int i = 0; i < kFrameSamples; ++i) {
|
||||
// Output must stay finite and within int16 range (clamping correctness).
|
||||
CHECK(frame[i] >= -32768 && frame[i] <= 32767);
|
||||
out_sumsq += static_cast<double>(frame[i]) * frame[i];
|
||||
}
|
||||
measured += kFrameSamples;
|
||||
}
|
||||
}
|
||||
|
||||
CHECK(measured > 0);
|
||||
double in_rms = std::sqrt(in_sumsq / measured);
|
||||
double out_rms = std::sqrt(out_sumsq / measured);
|
||||
double reduction = (in_rms > 0.0) ? (1.0 - out_rms / in_rms) : 0.0;
|
||||
std::printf("noise-only: in_rms=%.1f out_rms=%.1f reduction=%.1f%%\n", in_rms, out_rms,
|
||||
100.0 * reduction);
|
||||
|
||||
// RNNoise drops pure noise by ~99%; require a large, unambiguous reduction so a passthrough
|
||||
// (no real backend) is caught. The threshold is deliberately conservative vs. the ~99% seen.
|
||||
CHECK(reduction > 0.80);
|
||||
|
||||
// A 48-kHz guard miss must pass audio through untouched (our clock is always 48 kHz, but the
|
||||
// backstop matters): feed a non-48k sample-rate and confirm the buffer is unchanged.
|
||||
std::vector<int16_t> probe(kFrameSamples);
|
||||
for (int i = 0; i < kFrameSamples; ++i) probe[i] = next_noise();
|
||||
std::vector<int16_t> probe_copy = probe;
|
||||
ns->process_capture(probe.data(), kFrameSamples, 16000);
|
||||
CHECK(probe == probe_copy);
|
||||
|
||||
if (g_failures == 0) {
|
||||
std::printf("noise_suppression: OK\n");
|
||||
return 0;
|
||||
}
|
||||
std::printf("noise_suppression: %d failure(s)\n", g_failures);
|
||||
return 1;
|
||||
}
|
||||
Reference in New Issue
Block a user