feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback
Closes the three items PROGRESS.md's M3 section explicitly carried forward as
out of scope:
- Device enumeration (vc_list_devices) + input device selection
(vc_set_input_device), backed by AudioEngine::enumerate_devices() via
miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded
ma_device_id strings.
- VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk).
webrtc-audio-processing (the originally-planned APM) has no working
Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard
abseil-cpp dependency), so VAD is a new lightweight, dependency-free
energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor
interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it.
- True stereo playback: AudioEngine's mixer and output device now carry
stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded
stereo streams to mono before mixing.
- Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via
miniaudio's loopback device type), replacing test-only injection as the
production capture path.
Also: vccli gains --list-devices, --input-device, --input-mode, and
--share-screen-audio flags, plus a stdin command loop (ptt on/off, mode
vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all
four items (ABI-level + a white-box AudioEngine stereo-mix check).
Docs updated to match: voice.md, roadmap.md (decision-log entry superseding
the original webrtc-audio-processing choice), tech-stack.md, README.md,
architecture.md, CLAUDE.md, PROGRESS.md.
Still explicitly out of scope, documented not silently dropped: real
webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS
SCREEN_AUDIO capture, process-specific loopback, and a pre-existing
RT-thread rule violation in the capture path that predates this work.
Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev
and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive
standalone runs; manually verified live (vccli --list-devices against real
hardware, vccli --voice --input-mode vad streaming without incident).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
|
|
|
/*
|
|
|
|
|
* test_vad_ptt_devices — closes M3's "explicitly out of scope" gaps (PROGRESS.md): device
|
|
|
|
|
* enumeration, the VAD/PTT send-side input gate, and true stereo playback mixing.
|
|
|
|
|
*
|
|
|
|
|
* Mirrors test_m3_multistream.cpp's approach (real vc_client instances against a real
|
|
|
|
|
* in-process server, not raw sockets) for the ABI-level pieces, plus a white-box AudioEngine
|
|
|
|
|
* test for the stereo mixer (no audio hardware needed — see AudioEngine::mix_for_test).
|
|
|
|
|
*
|
|
|
|
|
* 1. Device enumeration (vc_list_devices) works pre-connect, for both kinds, and tolerates
|
|
|
|
|
* an empty list (headless CI build agents may have zero audio devices) — VC_OK is the
|
|
|
|
|
* only thing asserted, never count > 0.
|
|
|
|
|
* 2. VAD gate: under VC_INPUT_VOICE_ACTIVATION (the default), silent PCM never reaches the
|
|
|
|
|
* peer (no talking edge); loud PCM does.
|
|
|
|
|
* 3. PTT gate: under VC_INPUT_PUSH_TO_TALK, loud PCM is gated closed until
|
|
|
|
|
* vc_set_push_to_talk(1); then it reaches the peer.
|
|
|
|
|
* 4. Stereo playback mixer: white-box (AudioEngine directly) — a genuinely stereo decoded
|
|
|
|
|
* stream survives into the mix without being downmixed to mono.
|
|
|
|
|
*/
|
|
|
|
|
#include <cstdio>
|
|
|
|
|
|
|
|
|
|
#ifdef VOICECAT_HAS_NET
|
|
|
|
|
|
|
|
|
|
#include <atomic>
|
|
|
|
|
#include <chrono>
|
|
|
|
|
#include <cmath>
|
|
|
|
|
#include <condition_variable>
|
|
|
|
|
#include <cstdlib>
|
|
|
|
|
#include <filesystem>
|
|
|
|
|
#include <mutex>
|
|
|
|
|
#include <string>
|
|
|
|
|
#include <thread>
|
|
|
|
|
#include <vector>
|
|
|
|
|
|
|
|
|
|
#include "voicecat.h"
|
|
|
|
|
#include "server.h"
|
|
|
|
|
#include "db.h"
|
|
|
|
|
|
|
|
|
|
#ifdef VOICECAT_HAS_AUDIO
|
|
|
|
|
#include "audio/audio_engine.h"
|
|
|
|
|
#endif
|
|
|
|
|
#ifdef VOICECAT_HAS_OPUS
|
|
|
|
|
#include "codec/opus_codec.h"
|
|
|
|
|
#endif
|
|
|
|
|
|
|
|
|
|
// ── Event tracking (same shape as test_m3_multistream.cpp) ──────────────────────
|
|
|
|
|
|
|
|
|
|
struct TalkEvent {
|
|
|
|
|
uint32_t user_id;
|
|
|
|
|
uint32_t stream_id;
|
|
|
|
|
bool talking;
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
struct EventStore {
|
|
|
|
|
std::mutex mu;
|
|
|
|
|
std::condition_variable cv;
|
|
|
|
|
|
|
|
|
|
bool auth_ok{false};
|
|
|
|
|
uint32_t self_user_id{0};
|
|
|
|
|
bool channel_list_received{false};
|
|
|
|
|
bool saw_stream_started{false};
|
|
|
|
|
std::vector<TalkEvent> talk_events;
|
|
|
|
|
bool disconnected{false};
|
|
|
|
|
|
|
|
|
|
const char* label{nullptr};
|
feat(M4): Windows WinForms client, TOFU identity pinning, VAD threshold + always-on mode
Core ABI extensions (voicecat.h):
- vc_list_channels / vc_list_users / vc_list_user_streams — pull-based snapshot getters
for the channel-tree and user-list UI; session_model_mu_ guards cross-thread reads
- VC_EVENT_JOIN_RESULT / vc_join_channel — channel join with optional password
- VC_EVENT_SERVER_IDENTITY + vc_confirm_server_identity — TOFU gate that blocks io_thread_
until the UI approves or rejects; pins TLS leaf-cert SHA-256 (not declared Ed25519)
- vc_get_server_identity_display — Ed25519 fingerprint for human-readable display only
- VC_INPUT_ALWAYS_ON = 2 in vc_input_mode — transmit unconditionally, no VAD gate
- vc_set_vad_threshold — live RMS threshold update (0.0–1.0); EnergyVadProcessor stores
it atomically so the audio RT path reads without a lock
C++ implementation:
- SessionModel::apply_snapshot / apply_channel_event fixed to populate parent_id,
password_protected, and max_users (were permanently zeroed)
- TlsContext::peer_cert_fingerprint — SHA-256 of peer leaf cert DER via mbedTLS
- TofuStore split into peek (read-only) + pin (write) so first-connect only persists
after user approval; tofu_store_path in vc_config for per-user pin file location
- TcpAcceptor uses dual-stack IPv6+IPv4 fallback (fixes localhost → ::1 on Windows)
- windows-client CMake preset: Release shared DLL, static MinGW runtime, no tools/tests
- New C++ tests: test_channel_user_list_abi, test_tofu_flow (14/14 green)
Windows client (clients/windows/ — .NET 10 WinForms):
- VoiceCat.Interop: LibraryImport P/Invoke surface, UnmanagedCallersOnly callbacks,
Channel<VoiceCatEvent> event delivery drained by 30ms WinForms Timer
- VoiceCat.App: ConnectDialog (saved servers, DPAPI password storage), ServerIdentity-
Dialog (TOFU first-connect / mismatch warning), MainForm (channel TreeView, user
ListBox, RichTextBox chat, voice controls, device pickers, VAD/PTT/always-on mode,
per-user gain/mute/NR tuning, VAD sensitivity TrackBar, level meter ProgressBar)
- PttKeyCaptureDialog — focus-scoped PTT key capture (documented limitation)
- PerUserTuningDialog — real-time gain/mute/NR applied to all of a user's streams
- Accessibility: explicit AccessibleName/Description on every control, & mnemonics,
Activity log ListBox as durable screen-reader record, AutomationNotification for
curated live announcements
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:35:16 +02:00
|
|
|
|
|
|
|
|
// Set right after vc_client_create, before vc_connect — lets on_event auto-confirm the
|
|
|
|
|
// M4 TOFU gate (VC_EVENT_SERVER_IDENTITY below) for this headless test.
|
|
|
|
|
vc_client* client{nullptr};
|
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback
Closes the three items PROGRESS.md's M3 section explicitly carried forward as
out of scope:
- Device enumeration (vc_list_devices) + input device selection
(vc_set_input_device), backed by AudioEngine::enumerate_devices() via
miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded
ma_device_id strings.
- VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk).
webrtc-audio-processing (the originally-planned APM) has no working
Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard
abseil-cpp dependency), so VAD is a new lightweight, dependency-free
energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor
interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it.
- True stereo playback: AudioEngine's mixer and output device now carry
stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded
stereo streams to mono before mixing.
- Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via
miniaudio's loopback device type), replacing test-only injection as the
production capture path.
Also: vccli gains --list-devices, --input-device, --input-mode, and
--share-screen-audio flags, plus a stdin command loop (ptt on/off, mode
vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all
four items (ABI-level + a white-box AudioEngine stereo-mix check).
Docs updated to match: voice.md, roadmap.md (decision-log entry superseding
the original webrtc-audio-processing choice), tech-stack.md, README.md,
architecture.md, CLAUDE.md, PROGRESS.md.
Still explicitly out of scope, documented not silently dropped: real
webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS
SCREEN_AUDIO capture, process-specific loopback, and a pre-existing
RT-thread rule violation in the capture path that predates this work.
Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev
and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive
standalone runs; manually verified live (vccli --list-devices against real
hardware, vccli --voice --input-mode vad streaming without incident).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
|
|
|
};
|
|
|
|
|
|
|
|
|
|
static void on_event(void* user, const vc_event* ev) {
|
|
|
|
|
auto* s = static_cast<EventStore*>(user);
|
|
|
|
|
std::lock_guard lk(s->mu);
|
|
|
|
|
switch (ev->type) {
|
feat(M4): Windows WinForms client, TOFU identity pinning, VAD threshold + always-on mode
Core ABI extensions (voicecat.h):
- vc_list_channels / vc_list_users / vc_list_user_streams — pull-based snapshot getters
for the channel-tree and user-list UI; session_model_mu_ guards cross-thread reads
- VC_EVENT_JOIN_RESULT / vc_join_channel — channel join with optional password
- VC_EVENT_SERVER_IDENTITY + vc_confirm_server_identity — TOFU gate that blocks io_thread_
until the UI approves or rejects; pins TLS leaf-cert SHA-256 (not declared Ed25519)
- vc_get_server_identity_display — Ed25519 fingerprint for human-readable display only
- VC_INPUT_ALWAYS_ON = 2 in vc_input_mode — transmit unconditionally, no VAD gate
- vc_set_vad_threshold — live RMS threshold update (0.0–1.0); EnergyVadProcessor stores
it atomically so the audio RT path reads without a lock
C++ implementation:
- SessionModel::apply_snapshot / apply_channel_event fixed to populate parent_id,
password_protected, and max_users (were permanently zeroed)
- TlsContext::peer_cert_fingerprint — SHA-256 of peer leaf cert DER via mbedTLS
- TofuStore split into peek (read-only) + pin (write) so first-connect only persists
after user approval; tofu_store_path in vc_config for per-user pin file location
- TcpAcceptor uses dual-stack IPv6+IPv4 fallback (fixes localhost → ::1 on Windows)
- windows-client CMake preset: Release shared DLL, static MinGW runtime, no tools/tests
- New C++ tests: test_channel_user_list_abi, test_tofu_flow (14/14 green)
Windows client (clients/windows/ — .NET 10 WinForms):
- VoiceCat.Interop: LibraryImport P/Invoke surface, UnmanagedCallersOnly callbacks,
Channel<VoiceCatEvent> event delivery drained by 30ms WinForms Timer
- VoiceCat.App: ConnectDialog (saved servers, DPAPI password storage), ServerIdentity-
Dialog (TOFU first-connect / mismatch warning), MainForm (channel TreeView, user
ListBox, RichTextBox chat, voice controls, device pickers, VAD/PTT/always-on mode,
per-user gain/mute/NR tuning, VAD sensitivity TrackBar, level meter ProgressBar)
- PttKeyCaptureDialog — focus-scoped PTT key capture (documented limitation)
- PerUserTuningDialog — real-time gain/mute/NR applied to all of a user's streams
- Accessibility: explicit AccessibleName/Description on every control, & mnemonics,
Activity log ListBox as durable screen-reader record, AutomationNotification for
curated live announcements
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:35:16 +02:00
|
|
|
case VC_EVENT_SERVER_IDENTITY:
|
|
|
|
|
// No human to ask in a headless test — trust on first connect unconditionally.
|
|
|
|
|
vc_confirm_server_identity(s->client, 1);
|
|
|
|
|
break;
|
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback
Closes the three items PROGRESS.md's M3 section explicitly carried forward as
out of scope:
- Device enumeration (vc_list_devices) + input device selection
(vc_set_input_device), backed by AudioEngine::enumerate_devices() via
miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded
ma_device_id strings.
- VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk).
webrtc-audio-processing (the originally-planned APM) has no working
Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard
abseil-cpp dependency), so VAD is a new lightweight, dependency-free
energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor
interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it.
- True stereo playback: AudioEngine's mixer and output device now carry
stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded
stereo streams to mono before mixing.
- Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via
miniaudio's loopback device type), replacing test-only injection as the
production capture path.
Also: vccli gains --list-devices, --input-device, --input-mode, and
--share-screen-audio flags, plus a stdin command loop (ptt on/off, mode
vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all
four items (ABI-level + a white-box AudioEngine stereo-mix check).
Docs updated to match: voice.md, roadmap.md (decision-log entry superseding
the original webrtc-audio-processing choice), tech-stack.md, README.md,
architecture.md, CLAUDE.md, PROGRESS.md.
Still explicitly out of scope, documented not silently dropped: real
webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS
SCREEN_AUDIO capture, process-specific loopback, and a pre-existing
RT-thread rule violation in the capture path that predates this work.
Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev
and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive
standalone runs; manually verified live (vccli --list-devices against real
hardware, vccli --voice --input-mode vad streaming without incident).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
|
|
|
case VC_EVENT_AUTH_RESULT:
|
|
|
|
|
s->auth_ok = (ev->result == VC_OK);
|
|
|
|
|
s->self_user_id = ev->user_id;
|
|
|
|
|
break;
|
|
|
|
|
case VC_EVENT_CHANNEL_LIST:
|
|
|
|
|
s->channel_list_received = true;
|
|
|
|
|
break;
|
|
|
|
|
case VC_EVENT_STREAM_STARTED:
|
|
|
|
|
s->saw_stream_started = true;
|
|
|
|
|
break;
|
|
|
|
|
case VC_EVENT_TALK_STATE:
|
|
|
|
|
s->talk_events.push_back({ev->user_id, ev->stream_id, ev->u32a != 0});
|
|
|
|
|
break;
|
|
|
|
|
case VC_EVENT_DISCONNECTED:
|
|
|
|
|
s->disconnected = true;
|
|
|
|
|
break;
|
|
|
|
|
default:
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
s->cv.notify_all();
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
template <typename Pred>
|
|
|
|
|
static bool wait_for(EventStore& s, Pred pred, int timeout_ms) {
|
|
|
|
|
auto deadline = std::chrono::steady_clock::now() + std::chrono::milliseconds(timeout_ms);
|
|
|
|
|
std::unique_lock lk(s.mu);
|
|
|
|
|
return s.cv.wait_until(lk, deadline, [&] { return pred(s); });
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static std::vector<int16_t> make_sine_frame(int frame_idx, float freq_hz,
|
|
|
|
|
int frame_samples = 960) {
|
|
|
|
|
std::vector<int16_t> pcm(frame_samples);
|
|
|
|
|
for (int i = 0; i < frame_samples; ++i) {
|
|
|
|
|
float t = static_cast<float>(frame_idx * frame_samples + i) / 48000.0f;
|
|
|
|
|
pcm[i] = static_cast<int16_t>(std::sin(2.0f * 3.14159265f * freq_hz * t) * 16000.0f);
|
|
|
|
|
}
|
|
|
|
|
return pcm;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static std::vector<int16_t> make_silence_frame(int frame_samples = 960) {
|
|
|
|
|
return std::vector<int16_t>(frame_samples, 0);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// Did `talking==true` ever fire for (user_id, stream_id) at index >= `from`?
|
|
|
|
|
static bool saw_talking_true(EventStore& s, uint32_t user_id, uint32_t stream_id, size_t from) {
|
|
|
|
|
std::lock_guard lk(s.mu);
|
|
|
|
|
for (size_t i = from; i < s.talk_events.size(); ++i) {
|
|
|
|
|
auto& e = s.talk_events[i];
|
|
|
|
|
if (e.user_id == user_id && e.stream_id == stream_id && e.talking) return true;
|
|
|
|
|
}
|
|
|
|
|
return false;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static size_t talk_event_count(EventStore& s) {
|
|
|
|
|
std::lock_guard lk(s.mu);
|
|
|
|
|
return s.talk_events.size();
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// ── Test harness ──────────────────────────────────────────────────────────────
|
|
|
|
|
|
|
|
|
|
static int g_failures = 0;
|
|
|
|
|
#define CHECK(cond) \
|
|
|
|
|
do { \
|
|
|
|
|
if (!(cond)) { \
|
|
|
|
|
std::printf("FAIL: %s (%s:%d)\n", #cond, __FILE__, __LINE__); \
|
|
|
|
|
++g_failures; \
|
|
|
|
|
} \
|
|
|
|
|
} while (0)
|
|
|
|
|
|
|
|
|
|
// ── 1. Device enumeration (no server needed) ────────────────────────────────────
|
|
|
|
|
static void test_device_enumeration() {
|
|
|
|
|
vc_config cfg{"test-devices", "0.1", VC_LOG_OFF};
|
|
|
|
|
vc_callbacks cb{};
|
|
|
|
|
vc_client* c = vc_client_create(&cfg, cb);
|
|
|
|
|
CHECK(c != nullptr);
|
|
|
|
|
|
|
|
|
|
for (vc_device_kind kind : {VC_DEVICE_INPUT, VC_DEVICE_OUTPUT}) {
|
|
|
|
|
vc_device_list dl{};
|
|
|
|
|
vc_result r = vc_list_devices(c, kind, &dl);
|
|
|
|
|
#ifdef VOICECAT_HAS_AUDIO
|
|
|
|
|
CHECK(r == VC_OK);
|
|
|
|
|
// Headless CI build agents may legitimately report zero devices — never assert
|
|
|
|
|
// count > 0, only that the call itself succeeded and the list is well-formed.
|
|
|
|
|
for (size_t i = 0; i < dl.count; ++i) {
|
|
|
|
|
CHECK(dl.items[i].id != nullptr);
|
|
|
|
|
CHECK(dl.items[i].name != nullptr);
|
|
|
|
|
}
|
|
|
|
|
#else
|
|
|
|
|
CHECK(r == VC_ERR_NOT_IMPLEMENTED);
|
|
|
|
|
#endif
|
|
|
|
|
vc_free_device_list(&dl);
|
|
|
|
|
vc_free_device_list(&dl); // idempotent — must not crash on a second call
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
vc_client_destroy(c);
|
|
|
|
|
std::printf("test_device_enumeration: ok\n");
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// ── 4. Stereo playback mixer (white-box, no audio hardware needed) ──────────────
|
|
|
|
|
#if defined(VOICECAT_HAS_AUDIO) && defined(VOICECAT_HAS_OPUS)
|
|
|
|
|
static void test_stereo_mix() {
|
|
|
|
|
voicecat::audio::AudioEngine engine;
|
|
|
|
|
voicecat::audio::AudioParams p;
|
|
|
|
|
p.sample_rate = 48000;
|
|
|
|
|
p.capture_channels = 1;
|
|
|
|
|
p.playback_channels = 2;
|
|
|
|
|
p.frame_ms = 20;
|
|
|
|
|
CHECK(engine.start(p)); // capture_cb intentionally omitted — not exercised here
|
|
|
|
|
|
|
|
|
|
voicecat::codec::OpusParams stereo_params;
|
|
|
|
|
stereo_params.stereo = true;
|
|
|
|
|
int frame_samples = voicecat::codec::opus_frame_samples(stereo_params);
|
|
|
|
|
|
|
|
|
|
voicecat::codec::OpusEncoder enc;
|
|
|
|
|
CHECK(enc.init(stereo_params));
|
|
|
|
|
|
|
|
|
|
// Loud left channel, silent right channel — a real downmix would average them into a
|
|
|
|
|
// single audible-but-quieter centered sample; true stereo should keep them distinct.
|
|
|
|
|
std::vector<int16_t> interleaved(static_cast<size_t>(frame_samples) * 2);
|
|
|
|
|
for (int i = 0; i < frame_samples; ++i) {
|
|
|
|
|
float t = static_cast<float>(i) / 48000.0f;
|
|
|
|
|
interleaved[i * 2] = static_cast<int16_t>(std::sin(2.0f * 3.14159265f * 440.0f * t) * 20000.0f);
|
|
|
|
|
interleaved[i * 2 + 1] = 0;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
uint8_t opus_buf[1500];
|
|
|
|
|
int opus_len = enc.encode(interleaved.data(), frame_samples, opus_buf, sizeof(opus_buf));
|
|
|
|
|
CHECK(opus_len > 0);
|
|
|
|
|
|
|
|
|
|
engine.init_recv_stream(/*ssrc=*/1, stereo_params);
|
|
|
|
|
|
|
|
|
|
voicecat::audio::JitterBuffer::Frame f;
|
|
|
|
|
f.seq = 0;
|
|
|
|
|
f.timestamp = 0;
|
|
|
|
|
f.fec_present = false;
|
|
|
|
|
f.payload.assign(opus_buf, opus_buf + opus_len);
|
|
|
|
|
engine.push_recv_frame(1, std::move(f));
|
|
|
|
|
|
|
|
|
|
std::vector<int16_t> out(static_cast<size_t>(frame_samples) * 2, 0);
|
|
|
|
|
engine.mix_for_test(out.data(), static_cast<uint32_t>(frame_samples));
|
|
|
|
|
|
|
|
|
|
// If the engine downmixed (old M3 behavior), every L/R pair would be identical (the
|
|
|
|
|
// average of a loud sample and 0). True stereo should show a clear, consistent L != R
|
|
|
|
|
// difference across the frame.
|
|
|
|
|
int64_t total_diff = 0;
|
|
|
|
|
for (int i = 0; i < frame_samples; ++i)
|
|
|
|
|
total_diff += std::abs(static_cast<int>(out[i * 2]) - static_cast<int>(out[i * 2 + 1]));
|
|
|
|
|
CHECK(total_diff > static_cast<int64_t>(frame_samples) * 1000); // well above decode noise
|
|
|
|
|
|
|
|
|
|
engine.remove_stream(1);
|
|
|
|
|
engine.stop();
|
|
|
|
|
std::printf("test_stereo_mix: ok (total_diff=%lld)\n", static_cast<long long>(total_diff));
|
|
|
|
|
}
|
|
|
|
|
#endif // VOICECAT_HAS_AUDIO && VOICECAT_HAS_OPUS
|
|
|
|
|
|
|
|
|
|
// ── 2/3. VAD + PTT gate, through the real ABI against a real server ─────────────
|
|
|
|
|
static void test_vad_and_ptt_gate() {
|
|
|
|
|
auto tmp = std::filesystem::temp_directory_path() /
|
|
|
|
|
("vctest_vadptt_" + std::to_string(
|
|
|
|
|
std::chrono::steady_clock::now().time_since_epoch().count()));
|
|
|
|
|
std::filesystem::create_directories(tmp);
|
|
|
|
|
std::string data_dir = tmp.string();
|
|
|
|
|
|
|
|
|
|
std::atomic<uint16_t> bound_port{0};
|
|
|
|
|
std::mutex ready_mu;
|
|
|
|
|
std::condition_variable ready_cv;
|
|
|
|
|
bool ready{false};
|
|
|
|
|
|
|
|
|
|
voicecat::server::Config cfg;
|
|
|
|
|
cfg.data_dir = data_dir;
|
|
|
|
|
cfg.bind_port = 0;
|
|
|
|
|
cfg.media_port = 0;
|
|
|
|
|
cfg.server_name = "VoiceCat-VadPttTest";
|
|
|
|
|
cfg.allow_guests = true;
|
|
|
|
|
cfg.on_ready = [&](uint16_t p) {
|
|
|
|
|
bound_port.store(p);
|
|
|
|
|
{ std::lock_guard lk(ready_mu); ready = true; }
|
|
|
|
|
ready_cv.notify_all();
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
voicecat::server::Server server(cfg);
|
|
|
|
|
std::thread server_thread([&] { server.run(); });
|
|
|
|
|
|
|
|
|
|
{
|
|
|
|
|
std::unique_lock lk(ready_mu);
|
|
|
|
|
bool ok = ready_cv.wait_for(lk, std::chrono::seconds(10), [&] { return ready; });
|
|
|
|
|
if (!ok) {
|
|
|
|
|
std::printf("FAIL: server did not become ready within 10s\n");
|
|
|
|
|
++g_failures;
|
|
|
|
|
server.stop();
|
|
|
|
|
server_thread.join();
|
|
|
|
|
std::filesystem::remove_all(tmp);
|
|
|
|
|
return;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
uint16_t port = bound_port.load();
|
|
|
|
|
std::printf("test_vad_and_ptt_gate: server ready on :%u\n", port);
|
|
|
|
|
|
|
|
|
|
EventStore evA;
|
|
|
|
|
evA.label = "A";
|
|
|
|
|
vc_callbacks cbA{on_event, nullptr, &evA};
|
|
|
|
|
vc_config cfgA{"test-A", "0.1", VC_LOG_OFF};
|
|
|
|
|
vc_client* clientA = vc_client_create(&cfgA, cbA);
|
|
|
|
|
CHECK(clientA != nullptr);
|
feat(M4): Windows WinForms client, TOFU identity pinning, VAD threshold + always-on mode
Core ABI extensions (voicecat.h):
- vc_list_channels / vc_list_users / vc_list_user_streams — pull-based snapshot getters
for the channel-tree and user-list UI; session_model_mu_ guards cross-thread reads
- VC_EVENT_JOIN_RESULT / vc_join_channel — channel join with optional password
- VC_EVENT_SERVER_IDENTITY + vc_confirm_server_identity — TOFU gate that blocks io_thread_
until the UI approves or rejects; pins TLS leaf-cert SHA-256 (not declared Ed25519)
- vc_get_server_identity_display — Ed25519 fingerprint for human-readable display only
- VC_INPUT_ALWAYS_ON = 2 in vc_input_mode — transmit unconditionally, no VAD gate
- vc_set_vad_threshold — live RMS threshold update (0.0–1.0); EnergyVadProcessor stores
it atomically so the audio RT path reads without a lock
C++ implementation:
- SessionModel::apply_snapshot / apply_channel_event fixed to populate parent_id,
password_protected, and max_users (were permanently zeroed)
- TlsContext::peer_cert_fingerprint — SHA-256 of peer leaf cert DER via mbedTLS
- TofuStore split into peek (read-only) + pin (write) so first-connect only persists
after user approval; tofu_store_path in vc_config for per-user pin file location
- TcpAcceptor uses dual-stack IPv6+IPv4 fallback (fixes localhost → ::1 on Windows)
- windows-client CMake preset: Release shared DLL, static MinGW runtime, no tools/tests
- New C++ tests: test_channel_user_list_abi, test_tofu_flow (14/14 green)
Windows client (clients/windows/ — .NET 10 WinForms):
- VoiceCat.Interop: LibraryImport P/Invoke surface, UnmanagedCallersOnly callbacks,
Channel<VoiceCatEvent> event delivery drained by 30ms WinForms Timer
- VoiceCat.App: ConnectDialog (saved servers, DPAPI password storage), ServerIdentity-
Dialog (TOFU first-connect / mismatch warning), MainForm (channel TreeView, user
ListBox, RichTextBox chat, voice controls, device pickers, VAD/PTT/always-on mode,
per-user gain/mute/NR tuning, VAD sensitivity TrackBar, level meter ProgressBar)
- PttKeyCaptureDialog — focus-scoped PTT key capture (documented limitation)
- PerUserTuningDialog — real-time gain/mute/NR applied to all of a user's streams
- Accessibility: explicit AccessibleName/Description on every control, & mnemonics,
Activity log ListBox as durable screen-reader record, AutomationNotification for
curated live announcements
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:35:16 +02:00
|
|
|
evA.client = clientA;
|
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback
Closes the three items PROGRESS.md's M3 section explicitly carried forward as
out of scope:
- Device enumeration (vc_list_devices) + input device selection
(vc_set_input_device), backed by AudioEngine::enumerate_devices() via
miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded
ma_device_id strings.
- VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk).
webrtc-audio-processing (the originally-planned APM) has no working
Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard
abseil-cpp dependency), so VAD is a new lightweight, dependency-free
energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor
interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it.
- True stereo playback: AudioEngine's mixer and output device now carry
stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded
stereo streams to mono before mixing.
- Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via
miniaudio's loopback device type), replacing test-only injection as the
production capture path.
Also: vccli gains --list-devices, --input-device, --input-mode, and
--share-screen-audio flags, plus a stdin command loop (ptt on/off, mode
vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all
four items (ABI-level + a white-box AudioEngine stereo-mix check).
Docs updated to match: voice.md, roadmap.md (decision-log entry superseding
the original webrtc-audio-processing choice), tech-stack.md, README.md,
architecture.md, CLAUDE.md, PROGRESS.md.
Still explicitly out of scope, documented not silently dropped: real
webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS
SCREEN_AUDIO capture, process-specific loopback, and a pre-existing
RT-thread rule violation in the capture path that predates this work.
Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev
and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive
standalone runs; manually verified live (vccli --list-devices against real
hardware, vccli --voice --input-mode vad streaming without incident).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
|
|
|
CHECK(vc_connect(clientA, "127.0.0.1", port) == VC_OK);
|
|
|
|
|
CHECK(vc_authenticate_guest(clientA, "VP-A") == VC_OK);
|
|
|
|
|
CHECK(wait_for(evA, [](EventStore& s) { return s.auth_ok; }, 8000));
|
|
|
|
|
CHECK(wait_for(evA, [](EventStore& s) { return s.channel_list_received; }, 3000));
|
|
|
|
|
|
|
|
|
|
EventStore evB;
|
|
|
|
|
evB.label = "B";
|
|
|
|
|
vc_callbacks cbB{on_event, nullptr, &evB};
|
|
|
|
|
vc_config cfgB{"test-B", "0.1", VC_LOG_OFF};
|
|
|
|
|
vc_client* clientB = vc_client_create(&cfgB, cbB);
|
|
|
|
|
CHECK(clientB != nullptr);
|
feat(M4): Windows WinForms client, TOFU identity pinning, VAD threshold + always-on mode
Core ABI extensions (voicecat.h):
- vc_list_channels / vc_list_users / vc_list_user_streams — pull-based snapshot getters
for the channel-tree and user-list UI; session_model_mu_ guards cross-thread reads
- VC_EVENT_JOIN_RESULT / vc_join_channel — channel join with optional password
- VC_EVENT_SERVER_IDENTITY + vc_confirm_server_identity — TOFU gate that blocks io_thread_
until the UI approves or rejects; pins TLS leaf-cert SHA-256 (not declared Ed25519)
- vc_get_server_identity_display — Ed25519 fingerprint for human-readable display only
- VC_INPUT_ALWAYS_ON = 2 in vc_input_mode — transmit unconditionally, no VAD gate
- vc_set_vad_threshold — live RMS threshold update (0.0–1.0); EnergyVadProcessor stores
it atomically so the audio RT path reads without a lock
C++ implementation:
- SessionModel::apply_snapshot / apply_channel_event fixed to populate parent_id,
password_protected, and max_users (were permanently zeroed)
- TlsContext::peer_cert_fingerprint — SHA-256 of peer leaf cert DER via mbedTLS
- TofuStore split into peek (read-only) + pin (write) so first-connect only persists
after user approval; tofu_store_path in vc_config for per-user pin file location
- TcpAcceptor uses dual-stack IPv6+IPv4 fallback (fixes localhost → ::1 on Windows)
- windows-client CMake preset: Release shared DLL, static MinGW runtime, no tools/tests
- New C++ tests: test_channel_user_list_abi, test_tofu_flow (14/14 green)
Windows client (clients/windows/ — .NET 10 WinForms):
- VoiceCat.Interop: LibraryImport P/Invoke surface, UnmanagedCallersOnly callbacks,
Channel<VoiceCatEvent> event delivery drained by 30ms WinForms Timer
- VoiceCat.App: ConnectDialog (saved servers, DPAPI password storage), ServerIdentity-
Dialog (TOFU first-connect / mismatch warning), MainForm (channel TreeView, user
ListBox, RichTextBox chat, voice controls, device pickers, VAD/PTT/always-on mode,
per-user gain/mute/NR tuning, VAD sensitivity TrackBar, level meter ProgressBar)
- PttKeyCaptureDialog — focus-scoped PTT key capture (documented limitation)
- PerUserTuningDialog — real-time gain/mute/NR applied to all of a user's streams
- Accessibility: explicit AccessibleName/Description on every control, & mnemonics,
Activity log ListBox as durable screen-reader record, AutomationNotification for
curated live announcements
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:35:16 +02:00
|
|
|
evB.client = clientB;
|
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback
Closes the three items PROGRESS.md's M3 section explicitly carried forward as
out of scope:
- Device enumeration (vc_list_devices) + input device selection
(vc_set_input_device), backed by AudioEngine::enumerate_devices() via
miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded
ma_device_id strings.
- VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk).
webrtc-audio-processing (the originally-planned APM) has no working
Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard
abseil-cpp dependency), so VAD is a new lightweight, dependency-free
energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor
interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it.
- True stereo playback: AudioEngine's mixer and output device now carry
stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded
stereo streams to mono before mixing.
- Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via
miniaudio's loopback device type), replacing test-only injection as the
production capture path.
Also: vccli gains --list-devices, --input-device, --input-mode, and
--share-screen-audio flags, plus a stdin command loop (ptt on/off, mode
vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all
four items (ABI-level + a white-box AudioEngine stereo-mix check).
Docs updated to match: voice.md, roadmap.md (decision-log entry superseding
the original webrtc-audio-processing choice), tech-stack.md, README.md,
architecture.md, CLAUDE.md, PROGRESS.md.
Still explicitly out of scope, documented not silently dropped: real
webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS
SCREEN_AUDIO capture, process-specific loopback, and a pre-existing
RT-thread rule violation in the capture path that predates this work.
Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev
and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive
standalone runs; manually verified live (vccli --list-devices against real
hardware, vccli --voice --input-mode vad streaming without incident).
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
|
|
|
CHECK(vc_connect(clientB, "127.0.0.1", port) == VC_OK);
|
|
|
|
|
CHECK(vc_authenticate_guest(clientB, "VP-B") == VC_OK);
|
|
|
|
|
CHECK(wait_for(evB, [](EventStore& s) { return s.auth_ok; }, 8000));
|
|
|
|
|
CHECK(wait_for(evB, [](EventStore& s) { return s.channel_list_received; }, 3000));
|
|
|
|
|
|
|
|
|
|
uint32_t a_uid = 0;
|
|
|
|
|
{ std::lock_guard lk(evA.mu); a_uid = evA.self_user_id; }
|
|
|
|
|
|
|
|
|
|
std::this_thread::sleep_for(std::chrono::milliseconds(500));
|
|
|
|
|
|
|
|
|
|
vc_stream_desc mic_desc{};
|
|
|
|
|
mic_desc.kind = VC_STREAM_MIC;
|
|
|
|
|
mic_desc.label = "mic";
|
|
|
|
|
uint32_t mic_sid = 0;
|
|
|
|
|
CHECK(vc_stream_start(clientA, &mic_desc, &mic_sid) == VC_OK);
|
|
|
|
|
CHECK(wait_for(evB, [](EventStore& s) { return s.saw_stream_started; }, 5000));
|
|
|
|
|
CHECK(wait_for(evA, [](EventStore& s) { return s.saw_stream_started; }, 5000));
|
|
|
|
|
|
|
|
|
|
// ── 2a. VAD mode (default), silent PCM: must NOT reach B as a talking edge ──
|
|
|
|
|
CHECK(vc_set_input_mode(clientA, VC_INPUT_VOICE_ACTIVATION) == VC_OK);
|
|
|
|
|
for (int i = 0; i < 15; ++i) {
|
|
|
|
|
auto silence = make_silence_frame();
|
|
|
|
|
CHECK(vc_test_inject_capture(clientA, mic_sid, silence.data(), silence.size()) == VC_OK);
|
|
|
|
|
std::this_thread::sleep_for(std::chrono::milliseconds(20));
|
|
|
|
|
}
|
|
|
|
|
CHECK(!saw_talking_true(evB, a_uid, mic_sid, 0));
|
|
|
|
|
|
|
|
|
|
// ── 2b. VAD mode, loud PCM: must reach B as a talking edge ──────────────────
|
|
|
|
|
size_t mark = talk_event_count(evB);
|
|
|
|
|
for (int i = 0; i < 20; ++i) {
|
|
|
|
|
auto loud = make_sine_frame(i, 440.0f);
|
|
|
|
|
CHECK(vc_test_inject_capture(clientA, mic_sid, loud.data(), loud.size()) == VC_OK);
|
|
|
|
|
std::this_thread::sleep_for(std::chrono::milliseconds(20));
|
|
|
|
|
}
|
|
|
|
|
CHECK(wait_for(evB, [&](EventStore& s) {
|
|
|
|
|
for (size_t i = mark; i < s.talk_events.size(); ++i) {
|
|
|
|
|
auto& e = s.talk_events[i];
|
|
|
|
|
if (e.user_id == a_uid && e.stream_id == mic_sid && e.talking) return true;
|
|
|
|
|
}
|
|
|
|
|
return false;
|
|
|
|
|
}, 3000));
|
|
|
|
|
|
|
|
|
|
// ── 3a. PTT mode, key up: loud PCM must NOT reach B as a new talking edge ───
|
|
|
|
|
CHECK(vc_set_input_mode(clientA, VC_INPUT_PUSH_TO_TALK) == VC_OK);
|
|
|
|
|
CHECK(vc_set_push_to_talk(clientA, 0) == VC_OK);
|
|
|
|
|
// Let any in-flight VAD-driven talking state lapse (hang-time ~300ms) before measuring.
|
|
|
|
|
std::this_thread::sleep_for(std::chrono::milliseconds(500));
|
|
|
|
|
mark = talk_event_count(evB);
|
|
|
|
|
for (int i = 0; i < 20; ++i) {
|
|
|
|
|
auto loud = make_sine_frame(i, 440.0f);
|
|
|
|
|
CHECK(vc_test_inject_capture(clientA, mic_sid, loud.data(), loud.size()) == VC_OK);
|
|
|
|
|
std::this_thread::sleep_for(std::chrono::milliseconds(20));
|
|
|
|
|
}
|
|
|
|
|
std::this_thread::sleep_for(std::chrono::milliseconds(200));
|
|
|
|
|
CHECK(!saw_talking_true(evB, a_uid, mic_sid, mark));
|
|
|
|
|
|
|
|
|
|
// ── 3b. PTT mode, key down: loud PCM must reach B as a talking edge ─────────
|
|
|
|
|
CHECK(vc_set_push_to_talk(clientA, 1) == VC_OK);
|
|
|
|
|
mark = talk_event_count(evB);
|
|
|
|
|
for (int i = 0; i < 20; ++i) {
|
|
|
|
|
auto loud = make_sine_frame(i, 440.0f);
|
|
|
|
|
CHECK(vc_test_inject_capture(clientA, mic_sid, loud.data(), loud.size()) == VC_OK);
|
|
|
|
|
std::this_thread::sleep_for(std::chrono::milliseconds(20));
|
|
|
|
|
}
|
|
|
|
|
CHECK(wait_for(evB, [&](EventStore& s) {
|
|
|
|
|
for (size_t i = mark; i < s.talk_events.size(); ++i) {
|
|
|
|
|
auto& e = s.talk_events[i];
|
|
|
|
|
if (e.user_id == a_uid && e.stream_id == mic_sid && e.talking) return true;
|
|
|
|
|
}
|
|
|
|
|
return false;
|
|
|
|
|
}, 3000));
|
|
|
|
|
|
|
|
|
|
{ std::lock_guard lk(evA.mu); CHECK(!evA.disconnected); }
|
|
|
|
|
{ std::lock_guard lk(evB.mu); CHECK(!evB.disconnected); }
|
|
|
|
|
|
|
|
|
|
vc_disconnect(clientA);
|
|
|
|
|
vc_disconnect(clientB);
|
|
|
|
|
vc_client_destroy(clientA);
|
|
|
|
|
vc_client_destroy(clientB);
|
|
|
|
|
|
|
|
|
|
server.stop();
|
|
|
|
|
server_thread.join();
|
|
|
|
|
std::filesystem::remove_all(tmp);
|
|
|
|
|
|
|
|
|
|
std::printf("test_vad_and_ptt_gate: done\n");
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
int main() {
|
|
|
|
|
test_device_enumeration();
|
|
|
|
|
#if defined(VOICECAT_HAS_AUDIO) && defined(VOICECAT_HAS_OPUS)
|
|
|
|
|
test_stereo_mix();
|
|
|
|
|
#endif
|
|
|
|
|
test_vad_and_ptt_gate();
|
|
|
|
|
|
|
|
|
|
if (g_failures == 0) {
|
|
|
|
|
std::printf("vad_ptt_devices: all checks passed\n");
|
|
|
|
|
return 0;
|
|
|
|
|
}
|
|
|
|
|
std::printf("vad_ptt_devices: %d failure(s)\n", g_failures);
|
|
|
|
|
return 1;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#else // !VOICECAT_HAS_NET
|
|
|
|
|
|
|
|
|
|
int main() {
|
|
|
|
|
std::printf("vad_ptt_devices: SKIP (VOICECAT_HAS_NET not defined)\n");
|
|
|
|
|
return 0;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#endif // VOICECAT_HAS_NET
|