feat(audio): real noise suppression via vendored RNNoise (send + receive)
The two-sided NR plumbing (RemoteStream::recv_ns + the per-listener vc_set_remote_stream noise_reduction toggle) was wired but inert: ApmProcessor::create() returned a no-op passthrough, because the originally-planned webrtc-audio-processing has no working Windows/macOS build. Drop in RNNoise as the real backend behind the same ApmProcessor interface, lighting up both NR paths. - Vendor RNNoise (BSD-3 + CC0) at third_party/rnnoise/ — the vcpkg port is !windows !arm, so it can't cover our primary targets. Shrunk int8 model (78MB -> 11.7MB via upstream scripts/shrink_model.sh), built as a standalone C static lib with no RTCD (portable scalar path on x86, auto-NEON on arm64) under -DDISABLE_DEBUG_FLOAT. Model is baked in (rnnoise_create(NULL)); no runtime file. - New RnnoiseProcessor (core/src/audio/apm_processor.cpp) selected by ApmProcessor::create() when VOICECAT_HAS_NS. Mono/48kHz/480-sample; our clock is fixed 48kHz and Opus frame sizes are multiples of 480, so no resampling. RT-safe: allocates at construction, lock-free in the capture/playback callbacks. - Receive-side: lit up via the factory; gated to mono streams (a stereo stream is a screen-audio share, not voice). - Send-side (new): vc_set_input_noise_reduction(client, enable) ABI + vc_client::mic_ns_, run before input gain/VAD in on_capture_frame. A stereo mic is downmixed to mono ONLY when NR is on — with NR off a stereo mic keeps full stereo (never collapse mic quality unasked). - Enable C as a project language for the vendored lib. - New noise_suppression test: white noise through ApmProcessor::create() drops ~99.9% RMS. ctest --preset dev green, 28/28. windows-client DLL builds clean with vc_set_input_noise_reduction exported, system-only deps. - Docs synced: voice.md §10, tech-stack.md §1/§5, third_party/README.md, vcpkg.json note, PROGRESS.md, CLAUDE.md. Client on/off UI toggles (Windows/macOS/iOS) are the remaining follow-up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -75,6 +75,38 @@ if(VOICECAT_USE_VCPKG_DEPS)
|
||||
|
||||
target_compile_definitions(voicecat PUBLIC VOICECAT_HAS_OPUS VOICECAT_HAS_AUDIO)
|
||||
|
||||
# ── RNNoise — vendored noise-suppression DSP (third_party/rnnoise, BSD-3 + CC0). ──────────
|
||||
# The real backend behind ApmProcessor (docs/voice.md §10-11). Built as a standalone C
|
||||
# static lib with NO run-time CPU dispatch (RTCD off): portable scalar path on x86,
|
||||
# auto-NEON on arm64. -DDISABLE_DEBUG_FLOAT selects the int8-quantized weights that match our
|
||||
# shrunk model (third_party/README.md). The vcpkg port is !windows !arm, so we vendor it.
|
||||
set(RNNOISE_DIR ${CMAKE_CURRENT_SOURCE_DIR}/../third_party/rnnoise)
|
||||
add_library(rnnoise STATIC
|
||||
${RNNOISE_DIR}/src/denoise.c
|
||||
${RNNOISE_DIR}/src/rnn.c
|
||||
${RNNOISE_DIR}/src/pitch.c
|
||||
${RNNOISE_DIR}/src/kiss_fft.c
|
||||
${RNNOISE_DIR}/src/celt_lpc.c
|
||||
${RNNOISE_DIR}/src/nnet.c
|
||||
${RNNOISE_DIR}/src/nnet_default.c
|
||||
${RNNOISE_DIR}/src/parse_lpcnet_weights.c
|
||||
${RNNOISE_DIR}/src/rnnoise_data.c
|
||||
${RNNOISE_DIR}/src/rnnoise_tables.c)
|
||||
target_include_directories(rnnoise
|
||||
PUBLIC ${RNNOISE_DIR}/include
|
||||
PRIVATE ${RNNOISE_DIR}/src)
|
||||
target_compile_definitions(rnnoise PRIVATE DISABLE_DEBUG_FLOAT)
|
||||
set_target_properties(rnnoise PROPERTIES
|
||||
POSITION_INDEPENDENT_CODE ON # libvoicecat may be built SHARED (windows-client/apple)
|
||||
C_VISIBILITY_PRESET hidden)
|
||||
if(CMAKE_SYSTEM_NAME STREQUAL "iOS")
|
||||
# rnnoise is plain C; keep it compiling as C even though audio_engine.cpp is OBJCXX.
|
||||
set_source_files_properties(${RNNOISE_DIR}/src/rnnoise_data.c PROPERTIES LANGUAGE C)
|
||||
endif()
|
||||
|
||||
target_link_libraries(voicecat PRIVATE rnnoise)
|
||||
target_compile_definitions(voicecat PRIVATE VOICECAT_HAS_NS)
|
||||
|
||||
# On iOS, miniaudio's AVFoundation backend includes Objective-C headers (AVFoundation.h →
|
||||
# Foundation.h). Compiling those as plain C++ fails; setting LANGUAGE OBJCXX for
|
||||
# audio_engine.cpp (the only file that includes miniaudio.h directly) fixes this.
|
||||
|
||||
Reference in New Issue
Block a user