feat(audio): real noise suppression via vendored RNNoise (send + receive)
Some checks failed
Build Linux Binaries / linux/amd64 (push) Has been cancelled
Build Linux Binaries / linux/arm64 (push) Has been cancelled

The two-sided NR plumbing (RemoteStream::recv_ns + the per-listener
vc_set_remote_stream noise_reduction toggle) was wired but inert:
ApmProcessor::create() returned a no-op passthrough, because the
originally-planned webrtc-audio-processing has no working Windows/macOS
build. Drop in RNNoise as the real backend behind the same ApmProcessor
interface, lighting up both NR paths.

- Vendor RNNoise (BSD-3 + CC0) at third_party/rnnoise/ — the vcpkg port
  is !windows !arm, so it can't cover our primary targets. Shrunk int8
  model (78MB -> 11.7MB via upstream scripts/shrink_model.sh), built as a
  standalone C static lib with no RTCD (portable scalar path on x86,
  auto-NEON on arm64) under -DDISABLE_DEBUG_FLOAT. Model is baked in
  (rnnoise_create(NULL)); no runtime file.
- New RnnoiseProcessor (core/src/audio/apm_processor.cpp) selected by
  ApmProcessor::create() when VOICECAT_HAS_NS. Mono/48kHz/480-sample;
  our clock is fixed 48kHz and Opus frame sizes are multiples of 480, so
  no resampling. RT-safe: allocates at construction, lock-free in the
  capture/playback callbacks.
- Receive-side: lit up via the factory; gated to mono streams (a stereo
  stream is a screen-audio share, not voice).
- Send-side (new): vc_set_input_noise_reduction(client, enable) ABI +
  vc_client::mic_ns_, run before input gain/VAD in on_capture_frame. A
  stereo mic is downmixed to mono ONLY when NR is on — with NR off a
  stereo mic keeps full stereo (never collapse mic quality unasked).
- Enable C as a project language for the vendored lib.
- New noise_suppression test: white noise through ApmProcessor::create()
  drops ~99.9% RMS. ctest --preset dev green, 28/28. windows-client DLL
  builds clean with vc_set_input_noise_reduction exported, system-only deps.
- Docs synced: voice.md §10, tech-stack.md §1/§5, third_party/README.md,
  vcpkg.json note, PROGRESS.md, CLAUDE.md.

Client on/off UI toggles (Windows/macOS/iOS) are the remaining follow-up.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-06-23 13:30:54 +02:00
parent 7249a8fd30
commit bad9c7533a
50 changed files with 381395 additions and 23 deletions

34
third_party/README.md vendored Normal file
View File

@@ -0,0 +1,34 @@
# third_party/ — vendored dependencies
Dependencies that are **not** consumed through vcpkg live here, copied verbatim into the tree.
Everything here is permissively licensed (no GPL/LGPL) per the house rule in
[`docs/tech-stack.md`](../docs/tech-stack.md) §5.
## rnnoise/
Real-time speech **noise suppression** (the DSP backend behind `ApmProcessor`
see [`docs/voice.md`](../docs/voice.md) §1011). Used by the receive-side per-stream NR
(`RemoteStream::recv_ns`) and the send-side mic NR (`vc_client::mic_ns_`).
- **Upstream:** https://github.com/xiph/rnnoise
- **Vendored at commit:** `70f1d256acd4b34a572f999a05c87bf00b67730d`
- **License:** BSD-3-Clause (code, see `rnnoise/COPYING`) + CC0-1.0 (model weights).
- **Why vendored, not vcpkg:** the vcpkg `rnnoise` port is marked `!windows !arm`, i.e.
unavailable on our primary targets (Windows MinGW, Apple Silicon, iOS). RNNoise is small,
self-contained C99 with no dependencies, so we vendor it directly.
### What was copied / changed
- Only the **library** sources + headers (`src/*.c`, `src/*.h`, `src/x86/*.h`, `include/`).
The training/feature-dump tools (`dump_features.c`, `write_weights.c`, the `src/x86/*.c`
RTCD kernels), build scaffolding (autotools, Meson) and `torch/` `training/` dirs are omitted.
- `src/rnnoise_data.c` is the **shrunk** model: upstream's `scripts/shrink_model.sh` strips the
`#ifndef DISABLE_DEBUG_FLOAT` float-weight duplicates, taking the default model from ~78 MB to
~11.7 MB. We build with `-DDISABLE_DEBUG_FLOAT` so only the int8-quantized weights are used —
this is exactly how upstream's default (non-debug) build behaves. The model is the built-in
default loaded by `rnnoise_create(NULL)`; there is **no runtime model file**.
### Build
Built as a standalone static lib `rnnoise` in [`core/CMakeLists.txt`](../core/CMakeLists.txt)
(no RTCD; portable scalar path on x86, NEON on arm64), and linked into `libvoicecat` which
defines `VOICECAT_HAS_NS`. To refresh the model, re-run upstream `autogen.sh`/`download_model.sh`
+ `scripts/shrink_model.sh` and re-copy `src/rnnoise_data.{c,h}`.