Files
voice-cat/tests/test_vad_ptt_devices.cpp

733 lines
32 KiB
C++
Raw Normal View History

feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
/*
* test_vad_ptt_devices closes M3's "explicitly out of scope" gaps (PROGRESS.md): device
* enumeration, the VAD/PTT send-side input gate, and true stereo playback mixing.
*
* Mirrors test_m3_multistream.cpp's approach (real vc_client instances against a real
* in-process server, not raw sockets) for the ABI-level pieces, plus a white-box AudioEngine
* test for the stereo mixer (no audio hardware needed see AudioEngine::mix_for_test).
*
* 1. Device enumeration (vc_list_devices) works pre-connect, for both kinds, and tolerates
* an empty list (headless CI build agents may have zero audio devices) VC_OK is the
* only thing asserted, never count > 0.
* 2. VAD gate: under VC_INPUT_VOICE_ACTIVATION (the default), silent PCM never reaches the
* peer (no talking edge); loud PCM does.
* 3. PTT gate: under VC_INPUT_PUSH_TO_TALK, loud PCM is gated closed until
* vc_set_push_to_talk(1); then it reaches the peer.
* 4. Stereo playback mixer: white-box (AudioEngine directly) a genuinely stereo decoded
* stream survives into the mix without being downmixed to mono.
*/
#include <cstdio>
#include <atomic>
#include <chrono>
#include <cmath>
#include <condition_variable>
#include <cstdlib>
#include <filesystem>
#include <mutex>
#include <string>
#include <thread>
#include <vector>
#include "voicecat.h"
#include "server.h"
#include "db.h"
#include "audio/audio_engine.h"
#include "codec/opus_codec.h"
// ── Event tracking (same shape as test_m3_multistream.cpp) ──────────────────────
struct TalkEvent {
uint32_t user_id;
uint32_t stream_id;
bool talking;
};
struct EventStore {
std::mutex mu;
std::condition_variable cv;
bool auth_ok{false};
uint32_t self_user_id{0};
bool channel_list_received{false};
bool saw_stream_started{false};
std::vector<TalkEvent> talk_events;
feat: fix voice join/leave, channel edit defaults, channel-update stream restart Three bugs fixed across the full stack (proto/server/core/ABI/Win/macOS/iOS): 1. Join/Leave Voice now truly subscribes/unsubscribes from the voice plane. Previously the button only toggled the local mic — receiving was always on (gated by channel membership alone). Added a protocol-level voice subscription concept: new SubscribeVoiceRequest/UnsubscribeVoiceRequest/VoiceSubscriptionResult proto messages, User.voice_subscribed field, vc_join_voice/vc_leave_voice C ABI functions, VC_EVENT_VOICE_STATE event, server-side voice_subscribed flag checked by the SFU relay recipient filter, and core-client gating of remote-stream decoder setup. All three clients rewired to subscribe+mic on Join / unsubscribe on Leave. Text chat works regardless of voice subscription. 2. Channel edit dialog now shows the channel's actual current settings. The read struct vc_channel was missing sort_order and audio fields — only the write struct vc_channel_info had them. Extended vc_channel with both (additive, no ABI break), updated the session model and list_channels marshaling to populate them, and updated all three clients' edit callers to use actual channel info instead of hardcoded defaults. 3. Channel parameter updates now automatically restart everyone's streams. Previously editing a channel's audio config persisted and broadcast a ChannelEvent::UPDATED, but no layer restarted streams — encoders/decoders are frozen at announce time. handle_channel_event now detects audio-config changes on the user's current channel and stop->starts each active local stream. The server reads the updated config on re-announce; peers wire up fresh decoders at the new ssrc. All 29 CTest tests pass; Windows DLL + C# client build clean. Apple clients not yet compile-verified (Windows environment).
2026-06-24 14:29:39 +02:00
bool voice_subscribed{false};
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
bool disconnected{false};
const char* label{nullptr};
feat(M4): Windows WinForms client, TOFU identity pinning, VAD threshold + always-on mode Core ABI extensions (voicecat.h): - vc_list_channels / vc_list_users / vc_list_user_streams — pull-based snapshot getters for the channel-tree and user-list UI; session_model_mu_ guards cross-thread reads - VC_EVENT_JOIN_RESULT / vc_join_channel — channel join with optional password - VC_EVENT_SERVER_IDENTITY + vc_confirm_server_identity — TOFU gate that blocks io_thread_ until the UI approves or rejects; pins TLS leaf-cert SHA-256 (not declared Ed25519) - vc_get_server_identity_display — Ed25519 fingerprint for human-readable display only - VC_INPUT_ALWAYS_ON = 2 in vc_input_mode — transmit unconditionally, no VAD gate - vc_set_vad_threshold — live RMS threshold update (0.0–1.0); EnergyVadProcessor stores it atomically so the audio RT path reads without a lock C++ implementation: - SessionModel::apply_snapshot / apply_channel_event fixed to populate parent_id, password_protected, and max_users (were permanently zeroed) - TlsContext::peer_cert_fingerprint — SHA-256 of peer leaf cert DER via mbedTLS - TofuStore split into peek (read-only) + pin (write) so first-connect only persists after user approval; tofu_store_path in vc_config for per-user pin file location - TcpAcceptor uses dual-stack IPv6+IPv4 fallback (fixes localhost → ::1 on Windows) - windows-client CMake preset: Release shared DLL, static MinGW runtime, no tools/tests - New C++ tests: test_channel_user_list_abi, test_tofu_flow (14/14 green) Windows client (clients/windows/ — .NET 10 WinForms): - VoiceCat.Interop: LibraryImport P/Invoke surface, UnmanagedCallersOnly callbacks, Channel<VoiceCatEvent> event delivery drained by 30ms WinForms Timer - VoiceCat.App: ConnectDialog (saved servers, DPAPI password storage), ServerIdentity- Dialog (TOFU first-connect / mismatch warning), MainForm (channel TreeView, user ListBox, RichTextBox chat, voice controls, device pickers, VAD/PTT/always-on mode, per-user gain/mute/NR tuning, VAD sensitivity TrackBar, level meter ProgressBar) - PttKeyCaptureDialog — focus-scoped PTT key capture (documented limitation) - PerUserTuningDialog — real-time gain/mute/NR applied to all of a user's streams - Accessibility: explicit AccessibleName/Description on every control, & mnemonics, Activity log ListBox as durable screen-reader record, AutomationNotification for curated live announcements Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:35:16 +02:00
// Set right after vc_client_create, before vc_connect — lets on_event auto-confirm the
// M4 TOFU gate (VC_EVENT_SERVER_IDENTITY below) for this headless test.
vc_client* client{nullptr};
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
};
static void on_event(void* user, const vc_event* ev) {
auto* s = static_cast<EventStore*>(user);
std::lock_guard lk(s->mu);
switch (ev->type) {
feat(M4): Windows WinForms client, TOFU identity pinning, VAD threshold + always-on mode Core ABI extensions (voicecat.h): - vc_list_channels / vc_list_users / vc_list_user_streams — pull-based snapshot getters for the channel-tree and user-list UI; session_model_mu_ guards cross-thread reads - VC_EVENT_JOIN_RESULT / vc_join_channel — channel join with optional password - VC_EVENT_SERVER_IDENTITY + vc_confirm_server_identity — TOFU gate that blocks io_thread_ until the UI approves or rejects; pins TLS leaf-cert SHA-256 (not declared Ed25519) - vc_get_server_identity_display — Ed25519 fingerprint for human-readable display only - VC_INPUT_ALWAYS_ON = 2 in vc_input_mode — transmit unconditionally, no VAD gate - vc_set_vad_threshold — live RMS threshold update (0.0–1.0); EnergyVadProcessor stores it atomically so the audio RT path reads without a lock C++ implementation: - SessionModel::apply_snapshot / apply_channel_event fixed to populate parent_id, password_protected, and max_users (were permanently zeroed) - TlsContext::peer_cert_fingerprint — SHA-256 of peer leaf cert DER via mbedTLS - TofuStore split into peek (read-only) + pin (write) so first-connect only persists after user approval; tofu_store_path in vc_config for per-user pin file location - TcpAcceptor uses dual-stack IPv6+IPv4 fallback (fixes localhost → ::1 on Windows) - windows-client CMake preset: Release shared DLL, static MinGW runtime, no tools/tests - New C++ tests: test_channel_user_list_abi, test_tofu_flow (14/14 green) Windows client (clients/windows/ — .NET 10 WinForms): - VoiceCat.Interop: LibraryImport P/Invoke surface, UnmanagedCallersOnly callbacks, Channel<VoiceCatEvent> event delivery drained by 30ms WinForms Timer - VoiceCat.App: ConnectDialog (saved servers, DPAPI password storage), ServerIdentity- Dialog (TOFU first-connect / mismatch warning), MainForm (channel TreeView, user ListBox, RichTextBox chat, voice controls, device pickers, VAD/PTT/always-on mode, per-user gain/mute/NR tuning, VAD sensitivity TrackBar, level meter ProgressBar) - PttKeyCaptureDialog — focus-scoped PTT key capture (documented limitation) - PerUserTuningDialog — real-time gain/mute/NR applied to all of a user's streams - Accessibility: explicit AccessibleName/Description on every control, & mnemonics, Activity log ListBox as durable screen-reader record, AutomationNotification for curated live announcements Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:35:16 +02:00
case VC_EVENT_SERVER_IDENTITY:
// No human to ask in a headless test — trust on first connect unconditionally.
vc_confirm_server_identity(s->client, 1);
break;
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
case VC_EVENT_AUTH_RESULT:
s->auth_ok = (ev->result == VC_OK);
s->self_user_id = ev->user_id;
break;
case VC_EVENT_CHANNEL_LIST:
s->channel_list_received = true;
break;
feat: fix voice join/leave, channel edit defaults, channel-update stream restart Three bugs fixed across the full stack (proto/server/core/ABI/Win/macOS/iOS): 1. Join/Leave Voice now truly subscribes/unsubscribes from the voice plane. Previously the button only toggled the local mic — receiving was always on (gated by channel membership alone). Added a protocol-level voice subscription concept: new SubscribeVoiceRequest/UnsubscribeVoiceRequest/VoiceSubscriptionResult proto messages, User.voice_subscribed field, vc_join_voice/vc_leave_voice C ABI functions, VC_EVENT_VOICE_STATE event, server-side voice_subscribed flag checked by the SFU relay recipient filter, and core-client gating of remote-stream decoder setup. All three clients rewired to subscribe+mic on Join / unsubscribe on Leave. Text chat works regardless of voice subscription. 2. Channel edit dialog now shows the channel's actual current settings. The read struct vc_channel was missing sort_order and audio fields — only the write struct vc_channel_info had them. Extended vc_channel with both (additive, no ABI break), updated the session model and list_channels marshaling to populate them, and updated all three clients' edit callers to use actual channel info instead of hardcoded defaults. 3. Channel parameter updates now automatically restart everyone's streams. Previously editing a channel's audio config persisted and broadcast a ChannelEvent::UPDATED, but no layer restarted streams — encoders/decoders are frozen at announce time. handle_channel_event now detects audio-config changes on the user's current channel and stop->starts each active local stream. The server reads the updated config on re-announce; peers wire up fresh decoders at the new ssrc. All 29 CTest tests pass; Windows DLL + C# client build clean. Apple clients not yet compile-verified (Windows environment).
2026-06-24 14:29:39 +02:00
case VC_EVENT_VOICE_STATE:
s->voice_subscribed = (ev->u32a == 1);
break;
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
case VC_EVENT_STREAM_STARTED:
s->saw_stream_started = true;
break;
case VC_EVENT_TALK_STATE:
s->talk_events.push_back({ev->user_id, ev->stream_id, ev->u32a != 0});
break;
case VC_EVENT_DISCONNECTED:
s->disconnected = true;
break;
default:
break;
}
s->cv.notify_all();
}
template <typename Pred>
static bool wait_for(EventStore& s, Pred pred, int timeout_ms) {
auto deadline = std::chrono::steady_clock::now() + std::chrono::milliseconds(timeout_ms);
std::unique_lock lk(s.mu);
return s.cv.wait_until(lk, deadline, [&] { return pred(s); });
}
static std::vector<int16_t> make_sine_frame(int frame_idx, float freq_hz,
int frame_samples = 960) {
std::vector<int16_t> pcm(frame_samples);
for (int i = 0; i < frame_samples; ++i) {
float t = static_cast<float>(frame_idx * frame_samples + i) / 48000.0f;
pcm[i] = static_cast<int16_t>(std::sin(2.0f * 3.14159265f * freq_hz * t) * 16000.0f);
}
return pcm;
}
static std::vector<int16_t> make_silence_frame(int frame_samples = 960) {
return std::vector<int16_t>(frame_samples, 0);
}
// Did `talking==true` ever fire for (user_id, stream_id) at index >= `from`?
static bool saw_talking_true(EventStore& s, uint32_t user_id, uint32_t stream_id, size_t from) {
std::lock_guard lk(s.mu);
for (size_t i = from; i < s.talk_events.size(); ++i) {
auto& e = s.talk_events[i];
if (e.user_id == user_id && e.stream_id == stream_id && e.talking) return true;
}
return false;
}
static size_t talk_event_count(EventStore& s) {
std::lock_guard lk(s.mu);
return s.talk_events.size();
}
// ── Test harness ──────────────────────────────────────────────────────────────
static int g_failures = 0;
#define CHECK(cond) \
do { \
if (!(cond)) { \
std::printf("FAIL: %s (%s:%d)\n", #cond, __FILE__, __LINE__); \
++g_failures; \
} \
} while (0)
// ── 1. Device enumeration (no server needed) ────────────────────────────────────
static void test_device_enumeration() {
vc_config cfg{"test-devices", "0.1", VC_LOG_OFF};
vc_callbacks cb{};
vc_client* c = vc_client_create(&cfg, cb);
CHECK(c != nullptr);
for (vc_device_kind kind : {VC_DEVICE_INPUT, VC_DEVICE_OUTPUT}) {
vc_device_list dl{};
vc_result r = vc_list_devices(c, kind, &dl);
CHECK(r == VC_OK);
// Headless CI build agents may legitimately report zero devices — never assert
// count > 0, only that the call itself succeeded and the list is well-formed.
for (size_t i = 0; i < dl.count; ++i) {
CHECK(dl.items[i].id != nullptr);
CHECK(dl.items[i].name != nullptr);
}
vc_free_device_list(&dl);
vc_free_device_list(&dl); // idempotent — must not crash on a second call
}
vc_client_destroy(c);
std::printf("test_device_enumeration: ok\n");
}
// ── 4. Stereo playback mixer (white-box, no audio hardware needed) ──────────────
static void test_stereo_mix() {
voicecat::audio::AudioEngine engine;
voicecat::audio::AudioParams p;
p.sample_rate = 48000;
p.capture_channels = 1;
p.playback_channels = 2;
p.frame_ms = 20;
CHECK(engine.start(p)); // capture_cb intentionally omitted — not exercised here
voicecat::codec::OpusParams stereo_params;
stereo_params.stereo = true;
int frame_samples = voicecat::codec::opus_frame_samples(stereo_params);
voicecat::codec::OpusEncoder enc;
CHECK(enc.init(stereo_params));
// Loud left channel, silent right channel — a real downmix would average them into a
// single audible-but-quieter centered sample; true stereo should keep them distinct.
std::vector<int16_t> interleaved(static_cast<size_t>(frame_samples) * 2);
for (int i = 0; i < frame_samples; ++i) {
float t = static_cast<float>(i) / 48000.0f;
interleaved[i * 2] = static_cast<int16_t>(std::sin(2.0f * 3.14159265f * 440.0f * t) * 20000.0f);
interleaved[i * 2 + 1] = 0;
}
uint8_t opus_buf[1500];
int opus_len = enc.encode(interleaved.data(), frame_samples, opus_buf, sizeof(opus_buf));
CHECK(opus_len > 0);
engine.init_recv_stream(/*ssrc=*/1, stereo_params, /*user_id=*/0, /*stream_id=*/0,
/*is_voice=*/false);
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
voicecat::audio::JitterBuffer::Frame f;
f.seq = 0;
f.timestamp = 0;
f.fec_present = false;
f.payload.assign(opus_buf, opus_buf + opus_len);
engine.push_recv_frame(1, std::move(f));
std::vector<int16_t> out(static_cast<size_t>(frame_samples) * 2, 0);
engine.mix_for_test(out.data(), static_cast<uint32_t>(frame_samples));
// If the engine downmixed (old M3 behavior), every L/R pair would be identical (the
// average of a loud sample and 0). True stereo should show a clear, consistent L != R
// difference across the frame.
int64_t total_diff = 0;
for (int i = 0; i < frame_samples; ++i)
total_diff += std::abs(static_cast<int>(out[i * 2]) - static_cast<int>(out[i * 2 + 1]));
CHECK(total_diff > static_cast<int64_t>(frame_samples) * 1000); // well above decode noise
engine.remove_stream(1);
engine.stop();
std::printf("test_stereo_mix: ok (total_diff=%lld)\n", static_cast<long long>(total_diff));
}
// ── 4a-2. Stereo screen-audio loopback capture (white-box, no audio hardware needed) ──
// Regression for the mono-loopback bug: start_loopback_capture used to hardcode channels=1,
// downmixing the system's stereo mix to mono before the encoder ever saw it (and on_capture_frame
// then upmixed L=R to produce a fake-stereo bitstream). Now the loopback device opens in the
// channel's mode (stereo when the channel is stereo), so the encoder receives real interleaved
// L/R PCM and encodes it directly. This test drives feed_loopback_for_test with a loud-L /
// silent-R stereo signal, encodes it (as on_capture_frame now does for channels==2), decodes,
// and mixes — asserting L != R across the frame. A mono-downmixed-then-upmixed bitstream would
// have L == R. Mirrors test_stereo_mix but routes the encode side through the loopback
// accumulator path that the fix touches (feed_loopback_for_test → on_loopback's accumulator).
#if defined(VOICECAT_HAS_LOOPBACK)
static void test_loopback_stereo_capture() {
voicecat::audio::AudioEngine engine;
voicecat::audio::AudioParams p;
p.sample_rate = 48000;
p.capture_channels = 1; // mic path — irrelevant here; loopback has its own channel count
p.playback_channels = 2; // stereo mix output (for mix_for_test below)
p.frame_ms = 20;
CHECK(engine.start(p)); // no capture_cb — the real mic (if any) won't touch capture_accum_
voicecat::codec::OpusParams stereo_params;
stereo_params.stereo = true;
stereo_params.application = voicecat::codec::OpusApplication::Audio; // screen-audio channel
stereo_params.bitrate_bps = 128000; // music/screen-audio channel default
int frame_samples = voicecat::codec::opus_frame_samples(stereo_params);
voicecat::codec::OpusEncoder enc;
CHECK(enc.init(stereo_params));
// Loud left channel, silent right — a real mono downmix would average them into a single
// audible-but-quieter centered sample; true stereo keeps them distinct.
std::vector<int16_t> interleaved(static_cast<size_t>(frame_samples) * 2);
for (int i = 0; i < frame_samples; ++i) {
float t = static_cast<float>(i) / 48000.0f;
interleaved[i * 2] = static_cast<int16_t>(std::sin(2.0f * 3.14159265f * 440.0f * t) * 20000.0f);
interleaved[i * 2 + 1] = 0;
}
// Encode via the loopback accumulator path: feed_loopback_for_test drives on_loopback's
// accumulator and invokes the callback with channels=2 (the fix). The callback encodes
// exactly as on_capture_frame does for real-stereo SCREEN_AUDIO PCM — no upmix.
uint8_t opus_buf[1500];
int opus_len = 0;
int seen_channels = 0;
auto cb = [&](int /*kind*/, const int16_t* pcm, int /*samples*/, int channels) {
seen_channels = channels;
if (channels == 2) {
// The loopback accumulator must have preserved L/R distinctness pre-encode.
int64_t pre_diff = 0;
for (int i = 0; i < frame_samples; ++i)
pre_diff += std::abs(static_cast<int>(pcm[i * 2]) - static_cast<int>(pcm[i * 2 + 1]));
CHECK(pre_diff > static_cast<int64_t>(frame_samples) * 1000);
}
opus_len = enc.encode(pcm, frame_samples, opus_buf, sizeof(opus_buf));
};
engine.feed_loopback_for_test(interleaved.data(), frame_samples, 2, cb);
CHECK(seen_channels == 2); // the loopback path reported stereo, not downmixed mono
CHECK(opus_len > 0);
// Decode + mix — same recv path as test_stereo_mix. A real stereo bitstream should
// survive with L != R; a mono-downmixed-then-upmixed bitstream would have L == R.
engine.init_recv_stream(/*ssrc=*/3, stereo_params, /*user_id=*/0, /*stream_id=*/0,
/*is_voice=*/false);
voicecat::audio::JitterBuffer::Frame f;
f.seq = 0;
f.timestamp = 0;
f.fec_present = false;
f.payload.assign(opus_buf, opus_buf + opus_len);
engine.push_recv_frame(3, std::move(f));
std::vector<int16_t> out(static_cast<size_t>(frame_samples) * 2, 0);
engine.mix_for_test(out.data(), static_cast<uint32_t>(frame_samples));
int64_t total_diff = 0;
for (int i = 0; i < frame_samples; ++i)
total_diff += std::abs(static_cast<int>(out[i * 2]) - static_cast<int>(out[i * 2 + 1]));
CHECK(total_diff > static_cast<int64_t>(frame_samples) * 1000);
engine.remove_stream(3);
engine.stop();
std::printf("test_loopback_stereo_capture: ok (total_diff=%lld, seen_channels=%d)\n",
static_cast<long long>(total_diff), seen_channels);
}
#endif
// ── 4b. Playout-clock re-sync after a late join / silence gap ────────────────────
// Regression for the "talk indicator lit, no audio" bug: the playout clock free-runs (it
// advances every callback via PLC), while the sender's frame timestamps only advance while it
// is actually transmitting. After a silence gap or a late join the clock drifts past the jitter
// buffer's 500 ms late-drop window, so every real frame is dropped-as-late and the stream is
// permanently silent. on_playback must re-seed the clock to the earliest buffered frame.
static void test_playout_resync() {
voicecat::audio::AudioEngine engine;
voicecat::audio::AudioParams p;
p.sample_rate = 48000;
p.capture_channels = 1;
p.playback_channels = 2;
p.frame_ms = 20;
CHECK(engine.start(p));
voicecat::codec::OpusParams mono_params; // mono = the mic path
mono_params.stereo = false;
int frame_samples = voicecat::codec::opus_frame_samples(mono_params);
voicecat::codec::OpusEncoder enc;
CHECK(enc.init(mono_params));
std::vector<int16_t> sine(static_cast<size_t>(frame_samples));
for (int i = 0; i < frame_samples; ++i) {
float t = static_cast<float>(i) / 48000.0f;
sine[i] = static_cast<int16_t>(std::sin(2.0f * 3.14159265f * 440.0f * t) * 20000.0f);
}
uint8_t opus_buf[1500];
int opus_len = enc.encode(sine.data(), frame_samples, opus_buf, sizeof(opus_buf));
CHECK(opus_len > 0);
engine.init_recv_stream(/*ssrc=*/2, mono_params, /*user_id=*/0, /*stream_id=*/0,
/*is_voice=*/false);
std::vector<int16_t> out(static_cast<size_t>(frame_samples) * 2, 0);
// Free-run the playout clock with an empty jitter buffer (PLC every callback) far past the
// 500 ms late-drop window — this is what a silence gap / late join does in the field.
for (int i = 0; i < 100; ++i) // ~100 frames @ 20 ms = ~2 s, well past 500 ms
engine.mix_for_test(out.data(), static_cast<uint32_t>(frame_samples));
// Now a real frame arrives carrying a timestamp far behind the free-run clock. Without the
// re-sync it is dropped-as-late and playback stays silent; with it the clock snaps back and
// the frame is decoded and mixed.
voicecat::audio::JitterBuffer::Frame f;
f.seq = 0;
f.timestamp = 0; // stream-relative start, now far behind the drifted playout clock
f.fec_present = false;
f.payload.assign(opus_buf, opus_buf + opus_len);
engine.push_recv_frame(2, std::move(f));
std::fill(out.begin(), out.end(), 0);
engine.mix_for_test(out.data(), static_cast<uint32_t>(frame_samples));
int64_t energy = 0;
for (int16_t s : out) energy += std::abs(static_cast<int>(s));
CHECK(energy > static_cast<int64_t>(frame_samples) * 1000); // audible, not PLC silence
engine.remove_stream(2);
engine.stop();
std::printf("test_playout_resync: ok (energy=%lld)\n", static_cast<long long>(energy));
}
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
// ── 5. Capture-frame accumulation (white-box, no audio hardware needed) ──────────
// Regression for the capture-side analogue of the playback ring fix: miniaudio's capture
// callback fires at the hardware period (commonly 480 samples on WASAPI shared mode), while
// opus_encode() requires exactly frame_samples_ (960). Sub-frame chunks must be accumulated;
// the callback must receive exactly 960-sample frames regardless of input chunk size.
static void test_capture_frame_accumulation() {
voicecat::audio::AudioEngine engine;
voicecat::audio::AudioParams p;
p.sample_rate = 48000;
p.capture_channels = 1;
p.frame_ms = 20; // frame_samples_ = 960
std::atomic<int> call_count{0};
std::atomic<bool> wrong_size{false};
constexpr int kExpected = 960;
// Start WITHOUT a capture callback: the real mic (if any) fires on_capture(), but
// on_capture() returns immediately when capture_cb_ is null, so capture_accum_ is
// never touched by the hardware thread. feed_capture_for_test() bypasses capture_cb_
// and drives the same accumulator directly with the explicit `cb` below — no races.
CHECK(engine.start(p));
auto cb = [&](int /*kind*/, const int16_t* /*pcm*/, int samples, int /*channels*/) {
++call_count;
if (samples != kExpected) wrong_size.store(true);
};
// 480-sample (10 ms) input — WASAPI's common hardware period on modern Windows.
// Two 480-chunk inputs → exactly one callback at 960.
std::vector<int16_t> h(480, 1000);
engine.feed_capture_for_test(h.data(), 480, cb);
CHECK(call_count.load() == 0); // half a frame — no callback yet
engine.feed_capture_for_test(h.data(), 480, cb);
CHECK(call_count.load() == 1); // one full frame — callback fired once
// Mis-aligned split: 240 then 720 → still exactly one callback.
std::vector<int16_t> s(240, 500), l(720, 500);
engine.feed_capture_for_test(s.data(), 240, cb);
CHECK(call_count.load() == 1);
engine.feed_capture_for_test(l.data(), 720, cb);
CHECK(call_count.load() == 2);
// 1920-sample input (two Opus frames) → exactly two callbacks.
std::vector<int16_t> d(1920, 800);
engine.feed_capture_for_test(d.data(), 1920, cb);
CHECK(call_count.load() == 4);
CHECK(!wrong_size.load());
engine.stop();
std::printf("test_capture_frame_accumulation: ok (callbacks=%d)\n", call_count.load());
}
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
// ── 2/3. VAD + PTT gate, through the real ABI against a real server ─────────────
static void test_vad_and_ptt_gate() {
auto tmp = std::filesystem::temp_directory_path() /
("vctest_vadptt_" + std::to_string(
std::chrono::steady_clock::now().time_since_epoch().count()));
std::filesystem::create_directories(tmp);
std::string data_dir = tmp.string();
std::atomic<uint16_t> bound_port{0};
std::mutex ready_mu;
std::condition_variable ready_cv;
bool ready{false};
voicecat::server::Config cfg;
cfg.data_dir = data_dir;
cfg.bind_port = 0;
cfg.media_port = 0;
cfg.server_name = "VoiceCat-VadPttTest";
cfg.allow_guests = true;
cfg.on_ready = [&](uint16_t p) {
bound_port.store(p);
{ std::lock_guard lk(ready_mu); ready = true; }
ready_cv.notify_all();
};
voicecat::server::Server server(cfg);
std::thread server_thread([&] { server.run(); });
{
std::unique_lock lk(ready_mu);
bool ok = ready_cv.wait_for(lk, std::chrono::seconds(10), [&] { return ready; });
if (!ok) {
std::printf("FAIL: server did not become ready within 10s\n");
++g_failures;
server.stop();
server_thread.join();
std::filesystem::remove_all(tmp);
return;
}
}
uint16_t port = bound_port.load();
std::printf("test_vad_and_ptt_gate: server ready on :%u\n", port);
EventStore evA;
evA.label = "A";
vc_callbacks cbA{on_event, nullptr, &evA};
vc_config cfgA{"test-A", "0.1", VC_LOG_OFF};
vc_client* clientA = vc_client_create(&cfgA, cbA);
CHECK(clientA != nullptr);
feat(M4): Windows WinForms client, TOFU identity pinning, VAD threshold + always-on mode Core ABI extensions (voicecat.h): - vc_list_channels / vc_list_users / vc_list_user_streams — pull-based snapshot getters for the channel-tree and user-list UI; session_model_mu_ guards cross-thread reads - VC_EVENT_JOIN_RESULT / vc_join_channel — channel join with optional password - VC_EVENT_SERVER_IDENTITY + vc_confirm_server_identity — TOFU gate that blocks io_thread_ until the UI approves or rejects; pins TLS leaf-cert SHA-256 (not declared Ed25519) - vc_get_server_identity_display — Ed25519 fingerprint for human-readable display only - VC_INPUT_ALWAYS_ON = 2 in vc_input_mode — transmit unconditionally, no VAD gate - vc_set_vad_threshold — live RMS threshold update (0.0–1.0); EnergyVadProcessor stores it atomically so the audio RT path reads without a lock C++ implementation: - SessionModel::apply_snapshot / apply_channel_event fixed to populate parent_id, password_protected, and max_users (were permanently zeroed) - TlsContext::peer_cert_fingerprint — SHA-256 of peer leaf cert DER via mbedTLS - TofuStore split into peek (read-only) + pin (write) so first-connect only persists after user approval; tofu_store_path in vc_config for per-user pin file location - TcpAcceptor uses dual-stack IPv6+IPv4 fallback (fixes localhost → ::1 on Windows) - windows-client CMake preset: Release shared DLL, static MinGW runtime, no tools/tests - New C++ tests: test_channel_user_list_abi, test_tofu_flow (14/14 green) Windows client (clients/windows/ — .NET 10 WinForms): - VoiceCat.Interop: LibraryImport P/Invoke surface, UnmanagedCallersOnly callbacks, Channel<VoiceCatEvent> event delivery drained by 30ms WinForms Timer - VoiceCat.App: ConnectDialog (saved servers, DPAPI password storage), ServerIdentity- Dialog (TOFU first-connect / mismatch warning), MainForm (channel TreeView, user ListBox, RichTextBox chat, voice controls, device pickers, VAD/PTT/always-on mode, per-user gain/mute/NR tuning, VAD sensitivity TrackBar, level meter ProgressBar) - PttKeyCaptureDialog — focus-scoped PTT key capture (documented limitation) - PerUserTuningDialog — real-time gain/mute/NR applied to all of a user's streams - Accessibility: explicit AccessibleName/Description on every control, & mnemonics, Activity log ListBox as durable screen-reader record, AutomationNotification for curated live announcements Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:35:16 +02:00
evA.client = clientA;
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
CHECK(vc_connect(clientA, "127.0.0.1", port) == VC_OK);
CHECK(vc_authenticate_guest(clientA, "VP-A") == VC_OK);
CHECK(wait_for(evA, [](EventStore& s) { return s.auth_ok; }, 8000));
CHECK(wait_for(evA, [](EventStore& s) { return s.channel_list_received; }, 3000));
feat: fix voice join/leave, channel edit defaults, channel-update stream restart Three bugs fixed across the full stack (proto/server/core/ABI/Win/macOS/iOS): 1. Join/Leave Voice now truly subscribes/unsubscribes from the voice plane. Previously the button only toggled the local mic — receiving was always on (gated by channel membership alone). Added a protocol-level voice subscription concept: new SubscribeVoiceRequest/UnsubscribeVoiceRequest/VoiceSubscriptionResult proto messages, User.voice_subscribed field, vc_join_voice/vc_leave_voice C ABI functions, VC_EVENT_VOICE_STATE event, server-side voice_subscribed flag checked by the SFU relay recipient filter, and core-client gating of remote-stream decoder setup. All three clients rewired to subscribe+mic on Join / unsubscribe on Leave. Text chat works regardless of voice subscription. 2. Channel edit dialog now shows the channel's actual current settings. The read struct vc_channel was missing sort_order and audio fields — only the write struct vc_channel_info had them. Extended vc_channel with both (additive, no ABI break), updated the session model and list_channels marshaling to populate them, and updated all three clients' edit callers to use actual channel info instead of hardcoded defaults. 3. Channel parameter updates now automatically restart everyone's streams. Previously editing a channel's audio config persisted and broadcast a ChannelEvent::UPDATED, but no layer restarted streams — encoders/decoders are frozen at announce time. handle_channel_event now detects audio-config changes on the user's current channel and stop->starts each active local stream. The server reads the updated config on re-announce; peers wire up fresh decoders at the new ssrc. All 29 CTest tests pass; Windows DLL + C# client build clean. Apple clients not yet compile-verified (Windows environment).
2026-06-24 14:29:39 +02:00
CHECK(vc_join_voice(clientA) == VC_OK);
CHECK(wait_for(evA, [](EventStore& s) { return s.voice_subscribed; }, 5000));
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
EventStore evB;
evB.label = "B";
vc_callbacks cbB{on_event, nullptr, &evB};
vc_config cfgB{"test-B", "0.1", VC_LOG_OFF};
vc_client* clientB = vc_client_create(&cfgB, cbB);
CHECK(clientB != nullptr);
feat(M4): Windows WinForms client, TOFU identity pinning, VAD threshold + always-on mode Core ABI extensions (voicecat.h): - vc_list_channels / vc_list_users / vc_list_user_streams — pull-based snapshot getters for the channel-tree and user-list UI; session_model_mu_ guards cross-thread reads - VC_EVENT_JOIN_RESULT / vc_join_channel — channel join with optional password - VC_EVENT_SERVER_IDENTITY + vc_confirm_server_identity — TOFU gate that blocks io_thread_ until the UI approves or rejects; pins TLS leaf-cert SHA-256 (not declared Ed25519) - vc_get_server_identity_display — Ed25519 fingerprint for human-readable display only - VC_INPUT_ALWAYS_ON = 2 in vc_input_mode — transmit unconditionally, no VAD gate - vc_set_vad_threshold — live RMS threshold update (0.0–1.0); EnergyVadProcessor stores it atomically so the audio RT path reads without a lock C++ implementation: - SessionModel::apply_snapshot / apply_channel_event fixed to populate parent_id, password_protected, and max_users (were permanently zeroed) - TlsContext::peer_cert_fingerprint — SHA-256 of peer leaf cert DER via mbedTLS - TofuStore split into peek (read-only) + pin (write) so first-connect only persists after user approval; tofu_store_path in vc_config for per-user pin file location - TcpAcceptor uses dual-stack IPv6+IPv4 fallback (fixes localhost → ::1 on Windows) - windows-client CMake preset: Release shared DLL, static MinGW runtime, no tools/tests - New C++ tests: test_channel_user_list_abi, test_tofu_flow (14/14 green) Windows client (clients/windows/ — .NET 10 WinForms): - VoiceCat.Interop: LibraryImport P/Invoke surface, UnmanagedCallersOnly callbacks, Channel<VoiceCatEvent> event delivery drained by 30ms WinForms Timer - VoiceCat.App: ConnectDialog (saved servers, DPAPI password storage), ServerIdentity- Dialog (TOFU first-connect / mismatch warning), MainForm (channel TreeView, user ListBox, RichTextBox chat, voice controls, device pickers, VAD/PTT/always-on mode, per-user gain/mute/NR tuning, VAD sensitivity TrackBar, level meter ProgressBar) - PttKeyCaptureDialog — focus-scoped PTT key capture (documented limitation) - PerUserTuningDialog — real-time gain/mute/NR applied to all of a user's streams - Accessibility: explicit AccessibleName/Description on every control, & mnemonics, Activity log ListBox as durable screen-reader record, AutomationNotification for curated live announcements Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:35:16 +02:00
evB.client = clientB;
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
CHECK(vc_connect(clientB, "127.0.0.1", port) == VC_OK);
CHECK(vc_authenticate_guest(clientB, "VP-B") == VC_OK);
CHECK(wait_for(evB, [](EventStore& s) { return s.auth_ok; }, 8000));
CHECK(wait_for(evB, [](EventStore& s) { return s.channel_list_received; }, 3000));
feat: fix voice join/leave, channel edit defaults, channel-update stream restart Three bugs fixed across the full stack (proto/server/core/ABI/Win/macOS/iOS): 1. Join/Leave Voice now truly subscribes/unsubscribes from the voice plane. Previously the button only toggled the local mic — receiving was always on (gated by channel membership alone). Added a protocol-level voice subscription concept: new SubscribeVoiceRequest/UnsubscribeVoiceRequest/VoiceSubscriptionResult proto messages, User.voice_subscribed field, vc_join_voice/vc_leave_voice C ABI functions, VC_EVENT_VOICE_STATE event, server-side voice_subscribed flag checked by the SFU relay recipient filter, and core-client gating of remote-stream decoder setup. All three clients rewired to subscribe+mic on Join / unsubscribe on Leave. Text chat works regardless of voice subscription. 2. Channel edit dialog now shows the channel's actual current settings. The read struct vc_channel was missing sort_order and audio fields — only the write struct vc_channel_info had them. Extended vc_channel with both (additive, no ABI break), updated the session model and list_channels marshaling to populate them, and updated all three clients' edit callers to use actual channel info instead of hardcoded defaults. 3. Channel parameter updates now automatically restart everyone's streams. Previously editing a channel's audio config persisted and broadcast a ChannelEvent::UPDATED, but no layer restarted streams — encoders/decoders are frozen at announce time. handle_channel_event now detects audio-config changes on the user's current channel and stop->starts each active local stream. The server reads the updated config on re-announce; peers wire up fresh decoders at the new ssrc. All 29 CTest tests pass; Windows DLL + C# client build clean. Apple clients not yet compile-verified (Windows environment).
2026-06-24 14:29:39 +02:00
CHECK(vc_join_voice(clientB) == VC_OK);
CHECK(wait_for(evB, [](EventStore& s) { return s.voice_subscribed; }, 5000));
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
uint32_t a_uid = 0;
{ std::lock_guard lk(evA.mu); a_uid = evA.self_user_id; }
std::this_thread::sleep_for(std::chrono::milliseconds(500));
vc_stream_desc mic_desc{};
mic_desc.kind = VC_STREAM_MIC;
mic_desc.label = "mic";
uint32_t mic_sid = 0;
CHECK(vc_stream_start(clientA, &mic_desc, &mic_sid) == VC_OK);
CHECK(wait_for(evB, [](EventStore& s) { return s.saw_stream_started; }, 5000));
CHECK(wait_for(evA, [](EventStore& s) { return s.saw_stream_started; }, 5000));
// ── 2a. VAD mode (default), silent PCM: must NOT reach B as a talking edge ──
CHECK(vc_set_input_mode(clientA, VC_INPUT_VOICE_ACTIVATION) == VC_OK);
for (int i = 0; i < 15; ++i) {
auto silence = make_silence_frame();
CHECK(vc_test_inject_capture(clientA, mic_sid, silence.data(), silence.size()) == VC_OK);
std::this_thread::sleep_for(std::chrono::milliseconds(20));
}
CHECK(!saw_talking_true(evB, a_uid, mic_sid, 0));
// ── 2b. VAD mode, loud PCM: must reach B as a talking edge ──────────────────
size_t mark = talk_event_count(evB);
for (int i = 0; i < 20; ++i) {
auto loud = make_sine_frame(i, 440.0f);
CHECK(vc_test_inject_capture(clientA, mic_sid, loud.data(), loud.size()) == VC_OK);
std::this_thread::sleep_for(std::chrono::milliseconds(20));
}
CHECK(wait_for(evB, [&](EventStore& s) {
for (size_t i = mark; i < s.talk_events.size(); ++i) {
auto& e = s.talk_events[i];
if (e.user_id == a_uid && e.stream_id == mic_sid && e.talking) return true;
}
return false;
}, 3000));
// ── 3a. PTT mode, key up: loud PCM must NOT reach B as a new talking edge ───
CHECK(vc_set_input_mode(clientA, VC_INPUT_PUSH_TO_TALK) == VC_OK);
CHECK(vc_set_push_to_talk(clientA, 0) == VC_OK);
// Let any in-flight VAD-driven talking state lapse (hang-time ~300ms) before measuring.
std::this_thread::sleep_for(std::chrono::milliseconds(500));
mark = talk_event_count(evB);
for (int i = 0; i < 20; ++i) {
auto loud = make_sine_frame(i, 440.0f);
CHECK(vc_test_inject_capture(clientA, mic_sid, loud.data(), loud.size()) == VC_OK);
std::this_thread::sleep_for(std::chrono::milliseconds(20));
}
std::this_thread::sleep_for(std::chrono::milliseconds(200));
CHECK(!saw_talking_true(evB, a_uid, mic_sid, mark));
// ── 3b. PTT mode, key down: loud PCM must reach B as a talking edge ─────────
CHECK(vc_set_push_to_talk(clientA, 1) == VC_OK);
mark = talk_event_count(evB);
for (int i = 0; i < 20; ++i) {
auto loud = make_sine_frame(i, 440.0f);
CHECK(vc_test_inject_capture(clientA, mic_sid, loud.data(), loud.size()) == VC_OK);
std::this_thread::sleep_for(std::chrono::milliseconds(20));
}
CHECK(wait_for(evB, [&](EventStore& s) {
for (size_t i = mark; i < s.talk_events.size(); ++i) {
auto& e = s.talk_events[i];
if (e.user_id == a_uid && e.stream_id == mic_sid && e.talking) return true;
}
return false;
}, 3000));
{ std::lock_guard lk(evA.mu); CHECK(!evA.disconnected); }
{ std::lock_guard lk(evB.mu); CHECK(!evB.disconnected); }
vc_disconnect(clientA);
vc_disconnect(clientB);
vc_client_destroy(clientA);
vc_client_destroy(clientB);
server.stop();
server_thread.join();
std::filesystem::remove_all(tmp);
std::printf("test_vad_and_ptt_gate: done\n");
}
feat(ios): audio overhaul, Join/Leave Voice, channel-id sync fix, stereo mic capture Three iOS client problems fixed plus a new core stereo-mic capture ABI: 1. Channel-id sync bug (mic button permanently dimmed): SessionState never synced currentChannelId from the self user's channelId on connect, so the mic button (gated on currentChannelId == 0) stayed dimmed. Added syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522); called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult. Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState. 2. Join/Leave Voice button: replaced icon-only mic toggle with explicit text button (parity with macOS). Mute/deafen disable when not in voice. 3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input port selection, built-in mic orientation/polar patterns, Bluetooth HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay, UserDefaults persistence. AudioSessionManager delegates to it. 4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels() lets the core open the mic device in stereo (2-ch interleaved). LocalStream gains capture_channels; ensure_audio_running reads it; audio_engine.cpp capture_accum_ + on_capture updated to channel-aware accumulation. Test test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper VoiceCatClient.setCaptureChannels. 5. Settings UI rework: AVAudioSession-derived input/output tree replaces miniaudio device picker. 6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj). swift-tools-version 6.0 with swiftLanguageModes .v5. Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md updated; stale 'vc_audio_suspend/resume deferred' claims corrected. Verified: ctest --preset dev 21/21 green; swift test 6/6 green; xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
2026-06-19 13:17:52 +02:00
// ── 5. Stereo mic capture (vc_set_capture_channels) ───────────────────────────
// Verifies that the mic capture accumulator path handles stereo (channels=2) correctly:
// the accumulator is sized to frame_samples_*capture_channels, on_capture forwards the
// correct channel count, and the encoder receives real interleaved L/R PCM (not a mono
// downmix). Mirrors test_loopback_stereo_capture but routes through the mic capture
// accumulator (feed_capture_for_test with channels=2) instead of the loopback path.
// This is the headless CI test for the iOS stereo built-in mic feature (Part D).
static void test_stereo_mic_capture() {
voicecat::audio::AudioEngine engine;
voicecat::audio::AudioParams p;
p.sample_rate = 48000;
p.capture_channels = 2; // stereo mic capture (vc_set_capture_channels path)
p.playback_channels = 2; // stereo mix output (for mix_for_test below)
p.frame_ms = 20;
CHECK(engine.start(p)); // no capture_cb — the real mic (if any) won't touch capture_accum_
voicecat::codec::OpusParams stereo_params;
stereo_params.stereo = true;
stereo_params.application = voicecat::codec::OpusApplication::Voip; // mic stream
stereo_params.bitrate_bps = 64000; // mic default
int frame_samples = voicecat::codec::opus_frame_samples(stereo_params);
voicecat::codec::OpusEncoder enc;
CHECK(enc.init(stereo_params));
// Loud left channel, silent right — a mono downmix would average them; true stereo
// keeps them distinct (same signal as test_loopback_stereo_capture).
std::vector<int16_t> interleaved(static_cast<size_t>(frame_samples) * 2);
for (int i = 0; i < frame_samples; ++i) {
float t = static_cast<float>(i) / 48000.0f;
interleaved[i * 2] = static_cast<int16_t>(std::sin(2.0f * 3.14159265f * 440.0f * t) * 20000.0f);
interleaved[i * 2 + 1] = 0;
}
// Encode via the mic capture accumulator path: feed_capture_for_test with channels=2
// drives on_capture's accumulator and invokes the callback with channels=2. The callback
// encodes exactly as on_capture_frame does for channels==2 — direct stereo, no upmix.
uint8_t opus_buf[1500];
int opus_len = 0;
int seen_channels = 0;
auto cb = [&](int /*kind*/, const int16_t* pcm, int /*samples*/, int channels) {
seen_channels = channels;
if (channels == 2) {
// The capture accumulator must have preserved L/R distinctness pre-encode.
int64_t pre_diff = 0;
for (int i = 0; i < frame_samples; ++i)
pre_diff += std::abs(static_cast<int>(pcm[i * 2]) - static_cast<int>(pcm[i * 2 + 1]));
CHECK(pre_diff > static_cast<int64_t>(frame_samples) * 1000);
}
opus_len = enc.encode(pcm, frame_samples, opus_buf, sizeof(opus_buf));
};
engine.feed_capture_for_test(interleaved.data(), frame_samples, 2, cb);
CHECK(seen_channels == 2); // the mic capture path reported stereo, not mono
CHECK(opus_len > 0);
// Decode + mix — same recv path as test_stereo_mix. A real stereo bitstream should
// survive with L != R; a mono-downmixed-then-upmixed bitstream would have L == R.
engine.init_recv_stream(/*ssrc=*/5, stereo_params, /*user_id=*/0, /*stream_id=*/0,
/*is_voice=*/false);
feat(ios): audio overhaul, Join/Leave Voice, channel-id sync fix, stereo mic capture Three iOS client problems fixed plus a new core stereo-mic capture ABI: 1. Channel-id sync bug (mic button permanently dimmed): SessionState never synced currentChannelId from the self user's channelId on connect, so the mic button (gated on currentChannelId == 0) stayed dimmed. Added syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522); called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult. Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState. 2. Join/Leave Voice button: replaced icon-only mic toggle with explicit text button (parity with macOS). Mute/deafen disable when not in voice. 3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input port selection, built-in mic orientation/polar patterns, Bluetooth HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay, UserDefaults persistence. AudioSessionManager delegates to it. 4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels() lets the core open the mic device in stereo (2-ch interleaved). LocalStream gains capture_channels; ensure_audio_running reads it; audio_engine.cpp capture_accum_ + on_capture updated to channel-aware accumulation. Test test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper VoiceCatClient.setCaptureChannels. 5. Settings UI rework: AVAudioSession-derived input/output tree replaces miniaudio device picker. 6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj). swift-tools-version 6.0 with swiftLanguageModes .v5. Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md updated; stale 'vc_audio_suspend/resume deferred' claims corrected. Verified: ctest --preset dev 21/21 green; swift test 6/6 green; xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
2026-06-19 13:17:52 +02:00
voicecat::audio::JitterBuffer::Frame f;
f.seq = 0;
f.timestamp = 0;
f.fec_present = false;
f.payload.assign(opus_buf, opus_buf + opus_len);
engine.push_recv_frame(5, std::move(f));
std::vector<int16_t> out(static_cast<size_t>(frame_samples) * 2, 0);
engine.mix_for_test(out.data(), static_cast<uint32_t>(frame_samples));
int64_t total_diff = 0;
for (int i = 0; i < frame_samples; ++i)
total_diff += std::abs(static_cast<int>(out[i * 2]) - static_cast<int>(out[i * 2 + 1]));
CHECK(total_diff > static_cast<int64_t>(frame_samples) * 1000);
engine.remove_stream(5);
engine.stop();
std::printf("test_stereo_mic_capture: ok (total_diff=%lld, seen_channels=%d)\n",
static_cast<long long>(total_diff), seen_channels);
}
// ── 6. Stereo mic capture on a MONO channel (downmix safety) ──────────────────
// A stereo mic (vc_set_capture_channels=2) can be enabled while on a mono channel. The mic
// then delivers interleaved L/R, but the channel's Opus encoder is mono. encode_and_send_frame
// must fold L/R to mono before encoding — handing interleaved pairs straight to a mono
// opus_encode makes it read 2× the samples it should (wrong pitch / garbage). This mirrors that
// fold and proves the result is a valid mono bitstream that decodes to the expected averaged
// signal, rather than half-length junk.
static void test_stereo_mic_mono_channel() {
voicecat::codec::OpusParams mono_params;
mono_params.stereo = false; // mono channel — encoder is mono
mono_params.application = voicecat::codec::OpusApplication::Voip;
mono_params.bitrate_bps = 64000;
const int frame_samples = voicecat::codec::opus_frame_samples(mono_params);
voicecat::codec::OpusEncoder enc;
CHECK(enc.init(mono_params));
// Loud left, silent right — folding (L+R)/2 yields a half-amplitude tone on every sample.
std::vector<int16_t> interleaved(static_cast<size_t>(frame_samples) * 2);
for (int i = 0; i < frame_samples; ++i) {
float t = static_cast<float>(i) / 48000.0f;
interleaved[i * 2] = static_cast<int16_t>(std::sin(2.0f * 3.14159265f * 440.0f * t) * 20000.0f);
interleaved[i * 2 + 1] = 0;
}
// Fold exactly as encode_and_send_frame does for a stereo frame on a mono channel.
std::vector<int16_t> folded(frame_samples);
for (int i = 0; i < frame_samples; ++i)
folded[i] = static_cast<int16_t>(
(static_cast<int32_t>(interleaved[i * 2]) + static_cast<int32_t>(interleaved[i * 2 + 1])) / 2);
uint8_t opus_buf[1500];
int opus_len = enc.encode(folded.data(), frame_samples, opus_buf, sizeof(opus_buf));
CHECK(opus_len > 0);
// Decode mono and verify a full-length frame with real energy survived (a garbage half-read
// would either fail to decode the full frame_samples or come back near-silent / wrong length).
voicecat::codec::OpusDecoder dec;
CHECK(dec.init(mono_params));
std::vector<int16_t> decoded(frame_samples, 0);
int dec_samples = dec.decode(opus_buf, opus_len, decoded.data(), frame_samples);
CHECK(dec_samples == frame_samples);
int64_t energy = 0;
for (int i = 0; i < frame_samples; ++i) energy += std::abs(static_cast<int>(decoded[i]));
CHECK(energy > static_cast<int64_t>(frame_samples) * 500); // clearly audible, not silence
std::printf("test_stereo_mic_mono_channel: ok (opus_len=%d, energy=%lld)\n",
opus_len, static_cast<long long>(energy));
}
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
int main() {
test_device_enumeration();
test_stereo_mix();
#if defined(VOICECAT_HAS_LOOPBACK)
test_loopback_stereo_capture();
#endif
feat(ios): audio overhaul, Join/Leave Voice, channel-id sync fix, stereo mic capture Three iOS client problems fixed plus a new core stereo-mic capture ABI: 1. Channel-id sync bug (mic button permanently dimmed): SessionState never synced currentChannelId from the self user's channelId on connect, so the mic button (gated on currentChannelId == 0) stayed dimmed. Added syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522); called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult. Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState. 2. Join/Leave Voice button: replaced icon-only mic toggle with explicit text button (parity with macOS). Mute/deafen disable when not in voice. 3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input port selection, built-in mic orientation/polar patterns, Bluetooth HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay, UserDefaults persistence. AudioSessionManager delegates to it. 4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels() lets the core open the mic device in stereo (2-ch interleaved). LocalStream gains capture_channels; ensure_audio_running reads it; audio_engine.cpp capture_accum_ + on_capture updated to channel-aware accumulation. Test test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper VoiceCatClient.setCaptureChannels. 5. Settings UI rework: AVAudioSession-derived input/output tree replaces miniaudio device picker. 6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj). swift-tools-version 6.0 with swiftLanguageModes .v5. Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md updated; stale 'vc_audio_suspend/resume deferred' claims corrected. Verified: ctest --preset dev 21/21 green; swift test 6/6 green; xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
2026-06-19 13:17:52 +02:00
test_stereo_mic_capture();
test_stereo_mic_mono_channel();
test_playout_resync();
test_capture_frame_accumulation();
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
test_vad_and_ptt_gate();
if (g_failures == 0) {
std::printf("vad_ptt_devices: all checks passed\n");
return 0;
}
std::printf("vad_ptt_devices: %d failure(s)\n", g_failures);
return 1;
}