Talon
bad9c7533a
The two-sided NR plumbing (RemoteStream::recv_ns + the per-listener vc_set_remote_stream noise_reduction toggle) was wired but inert: ApmProcessor::create() returned a no-op passthrough, because the originally-planned webrtc-audio-processing has no working Windows/macOS build. Drop in RNNoise as the real backend behind the same ApmProcessor interface, lighting up both NR paths. - Vendor RNNoise (BSD-3 + CC0) at third_party/rnnoise/ — the vcpkg port is !windows !arm, so it can't cover our primary targets. Shrunk int8 model (78MB -> 11.7MB via upstream scripts/shrink_model.sh), built as a standalone C static lib with no RTCD (portable scalar path on x86, auto-NEON on arm64) under -DDISABLE_DEBUG_FLOAT. Model is baked in (rnnoise_create(NULL)); no runtime file. - New RnnoiseProcessor (core/src/audio/apm_processor.cpp) selected by ApmProcessor::create() when VOICECAT_HAS_NS. Mono/48kHz/480-sample; our clock is fixed 48kHz and Opus frame sizes are multiples of 480, so no resampling. RT-safe: allocates at construction, lock-free in the capture/playback callbacks. - Receive-side: lit up via the factory; gated to mono streams (a stereo stream is a screen-audio share, not voice). - Send-side (new): vc_set_input_noise_reduction(client, enable) ABI + vc_client::mic_ns_, run before input gain/VAD in on_capture_frame. A stereo mic is downmixed to mono ONLY when NR is on — with NR off a stereo mic keeps full stereo (never collapse mic quality unasked). - Enable C as a project language for the vendored lib. - New noise_suppression test: white noise through ApmProcessor::create() drops ~99.9% RMS. ctest --preset dev green, 28/28. windows-client DLL builds clean with vc_set_input_noise_reduction exported, system-only deps. - Docs synced: voice.md §10, tech-stack.md §1/§5, third_party/README.md, vcpkg.json note, PROGRESS.md, CLAUDE.md. Client on/off UI toggles (Windows/macOS/iOS) are the remaining follow-up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
35 lines
2.1 KiB
Markdown
35 lines
2.1 KiB
Markdown
# third_party/ — vendored dependencies
|
||
|
||
Dependencies that are **not** consumed through vcpkg live here, copied verbatim into the tree.
|
||
Everything here is permissively licensed (no GPL/LGPL) per the house rule in
|
||
[`docs/tech-stack.md`](../docs/tech-stack.md) §5.
|
||
|
||
## rnnoise/
|
||
|
||
Real-time speech **noise suppression** (the DSP backend behind `ApmProcessor` —
|
||
see [`docs/voice.md`](../docs/voice.md) §10–11). Used by the receive-side per-stream NR
|
||
(`RemoteStream::recv_ns`) and the send-side mic NR (`vc_client::mic_ns_`).
|
||
|
||
- **Upstream:** https://github.com/xiph/rnnoise
|
||
- **Vendored at commit:** `70f1d256acd4b34a572f999a05c87bf00b67730d`
|
||
- **License:** BSD-3-Clause (code, see `rnnoise/COPYING`) + CC0-1.0 (model weights).
|
||
- **Why vendored, not vcpkg:** the vcpkg `rnnoise` port is marked `!windows !arm`, i.e.
|
||
unavailable on our primary targets (Windows MinGW, Apple Silicon, iOS). RNNoise is small,
|
||
self-contained C99 with no dependencies, so we vendor it directly.
|
||
|
||
### What was copied / changed
|
||
- Only the **library** sources + headers (`src/*.c`, `src/*.h`, `src/x86/*.h`, `include/`).
|
||
The training/feature-dump tools (`dump_features.c`, `write_weights.c`, the `src/x86/*.c`
|
||
RTCD kernels), build scaffolding (autotools, Meson) and `torch/` `training/` dirs are omitted.
|
||
- `src/rnnoise_data.c` is the **shrunk** model: upstream's `scripts/shrink_model.sh` strips the
|
||
`#ifndef DISABLE_DEBUG_FLOAT` float-weight duplicates, taking the default model from ~78 MB to
|
||
~11.7 MB. We build with `-DDISABLE_DEBUG_FLOAT` so only the int8-quantized weights are used —
|
||
this is exactly how upstream's default (non-debug) build behaves. The model is the built-in
|
||
default loaded by `rnnoise_create(NULL)`; there is **no runtime model file**.
|
||
|
||
### Build
|
||
Built as a standalone static lib `rnnoise` in [`core/CMakeLists.txt`](../core/CMakeLists.txt)
|
||
(no RTCD; portable scalar path on x86, NEON on arm64), and linked into `libvoicecat` which
|
||
defines `VOICECAT_HAS_NS`. To refresh the model, re-run upstream `autogen.sh`/`download_model.sh`
|
||
+ `scripts/shrink_model.sh` and re-copy `src/rnnoise_data.{c,h}`.
|