feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback
Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -1,7 +1,18 @@
|
||||
#include "audio/apm_processor.h"
|
||||
|
||||
#include <chrono>
|
||||
#include <cmath>
|
||||
|
||||
namespace voicecat::audio {
|
||||
|
||||
namespace {
|
||||
int64_t steady_now_ms() {
|
||||
return std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||
std::chrono::steady_clock::now().time_since_epoch())
|
||||
.count();
|
||||
}
|
||||
} // namespace
|
||||
|
||||
// ── ApmPassthrough ────────────────────────────────────────────────────────────
|
||||
// No-op: returns true (VAD always open), does not modify PCM.
|
||||
// Replaced by WebrtcApmProcessor when VOICECAT_HAS_APM is defined.
|
||||
@@ -11,11 +22,47 @@ class ApmPassthrough final : public ApmProcessor {
|
||||
bool process_capture(int16_t*, int, int) override { return true; }
|
||||
};
|
||||
|
||||
// ── EnergyVadProcessor ──────────────────────────────────────────────────────
|
||||
// Lightweight, dependency-free energy/RMS VAD — see apm_processor.h's create_vad() doc comment
|
||||
// for why this exists instead of a real APM. No AEC (process_render is a no-op); doesn't modify
|
||||
// the PCM it's given, only inspects it.
|
||||
class EnergyVadProcessor final : public ApmProcessor {
|
||||
public:
|
||||
EnergyVadProcessor(float rms_threshold, int64_t hang_time_ms)
|
||||
: threshold_(rms_threshold), hang_time_ms_(hang_time_ms) {}
|
||||
|
||||
void process_render(const int16_t*, int, int) override {}
|
||||
|
||||
bool process_capture(int16_t* pcm, int samples, int /*sample_rate*/) override {
|
||||
if (samples > 0) {
|
||||
double sum_sq = 0.0;
|
||||
for (int i = 0; i < samples; ++i) {
|
||||
double s = static_cast<double>(pcm[i]) / 32768.0;
|
||||
sum_sq += s * s;
|
||||
}
|
||||
double rms = std::sqrt(sum_sq / samples);
|
||||
if (rms >= threshold_) last_voice_ms_ = steady_now_ms();
|
||||
}
|
||||
return (steady_now_ms() - last_voice_ms_) < hang_time_ms_;
|
||||
}
|
||||
|
||||
private:
|
||||
float threshold_;
|
||||
int64_t hang_time_ms_;
|
||||
int64_t last_voice_ms_ = 0; // epoch start -> gate begins closed until first loud frame
|
||||
};
|
||||
|
||||
std::unique_ptr<ApmProcessor> ApmProcessor::create() {
|
||||
#ifdef VOICECAT_HAS_APM
|
||||
// TODO(M3): return std::make_unique<WebrtcApmProcessor>();
|
||||
// TODO: return std::make_unique<WebrtcApmProcessor>(); — see create_vad()'s doc comment for
|
||||
// why this isn't wired up yet (no working Windows/MSVC build upstream).
|
||||
#endif
|
||||
return std::make_unique<ApmPassthrough>();
|
||||
}
|
||||
|
||||
std::unique_ptr<ApmProcessor> ApmProcessor::create_vad(float rms_threshold,
|
||||
int64_t hang_time_ms) {
|
||||
return std::make_unique<EnergyVadProcessor>(rms_threshold, hang_time_ms);
|
||||
}
|
||||
|
||||
} // namespace voicecat::audio
|
||||
|
||||
Reference in New Issue
Block a user