Files
voice-cat/tests/test_vad_ptt_devices.cpp

696 lines
30 KiB
C++
Raw Permalink Normal View History

feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
/*
* test_vad_ptt_devices closes M3's "explicitly out of scope" gaps (PROGRESS.md): device
* enumeration, the VAD/PTT send-side input gate, and true stereo playback mixing.
*
* Mirrors test_m3_multistream.cpp's approach (real vc_client instances against a real
* in-process server, not raw sockets) for the ABI-level pieces, plus a white-box AudioEngine
* test for the stereo mixer (no audio hardware needed see AudioEngine::mix_for_test).
*
* 1. Device enumeration (vc_list_devices) works pre-connect, for both kinds, and tolerates
* an empty list (headless CI build agents may have zero audio devices) VC_OK is the
* only thing asserted, never count > 0.
* 2. VAD gate: under VC_INPUT_VOICE_ACTIVATION (the default), silent PCM never reaches the
* peer (no talking edge); loud PCM does.
* 3. PTT gate: under VC_INPUT_PUSH_TO_TALK, loud PCM is gated closed until
* vc_set_push_to_talk(1); then it reaches the peer.
* 4. Stereo playback mixer: white-box (AudioEngine directly) a genuinely stereo decoded
* stream survives into the mix without being downmixed to mono.
*/
#include <cstdio>
#ifdef VOICECAT_HAS_NET
#include <atomic>
#include <chrono>
#include <cmath>
#include <condition_variable>
#include <cstdlib>
#include <filesystem>
#include <mutex>
#include <string>
#include <thread>
#include <vector>
#include "voicecat.h"
#include "server.h"
#include "db.h"
#ifdef VOICECAT_HAS_AUDIO
#include "audio/audio_engine.h"
#endif
#ifdef VOICECAT_HAS_OPUS
#include "codec/opus_codec.h"
#endif
// ── Event tracking (same shape as test_m3_multistream.cpp) ──────────────────────
struct TalkEvent {
uint32_t user_id;
uint32_t stream_id;
bool talking;
};
struct EventStore {
std::mutex mu;
std::condition_variable cv;
bool auth_ok{false};
uint32_t self_user_id{0};
bool channel_list_received{false};
bool saw_stream_started{false};
std::vector<TalkEvent> talk_events;
bool disconnected{false};
const char* label{nullptr};
feat(M4): Windows WinForms client, TOFU identity pinning, VAD threshold + always-on mode Core ABI extensions (voicecat.h): - vc_list_channels / vc_list_users / vc_list_user_streams — pull-based snapshot getters for the channel-tree and user-list UI; session_model_mu_ guards cross-thread reads - VC_EVENT_JOIN_RESULT / vc_join_channel — channel join with optional password - VC_EVENT_SERVER_IDENTITY + vc_confirm_server_identity — TOFU gate that blocks io_thread_ until the UI approves or rejects; pins TLS leaf-cert SHA-256 (not declared Ed25519) - vc_get_server_identity_display — Ed25519 fingerprint for human-readable display only - VC_INPUT_ALWAYS_ON = 2 in vc_input_mode — transmit unconditionally, no VAD gate - vc_set_vad_threshold — live RMS threshold update (0.0–1.0); EnergyVadProcessor stores it atomically so the audio RT path reads without a lock C++ implementation: - SessionModel::apply_snapshot / apply_channel_event fixed to populate parent_id, password_protected, and max_users (were permanently zeroed) - TlsContext::peer_cert_fingerprint — SHA-256 of peer leaf cert DER via mbedTLS - TofuStore split into peek (read-only) + pin (write) so first-connect only persists after user approval; tofu_store_path in vc_config for per-user pin file location - TcpAcceptor uses dual-stack IPv6+IPv4 fallback (fixes localhost → ::1 on Windows) - windows-client CMake preset: Release shared DLL, static MinGW runtime, no tools/tests - New C++ tests: test_channel_user_list_abi, test_tofu_flow (14/14 green) Windows client (clients/windows/ — .NET 10 WinForms): - VoiceCat.Interop: LibraryImport P/Invoke surface, UnmanagedCallersOnly callbacks, Channel<VoiceCatEvent> event delivery drained by 30ms WinForms Timer - VoiceCat.App: ConnectDialog (saved servers, DPAPI password storage), ServerIdentity- Dialog (TOFU first-connect / mismatch warning), MainForm (channel TreeView, user ListBox, RichTextBox chat, voice controls, device pickers, VAD/PTT/always-on mode, per-user gain/mute/NR tuning, VAD sensitivity TrackBar, level meter ProgressBar) - PttKeyCaptureDialog — focus-scoped PTT key capture (documented limitation) - PerUserTuningDialog — real-time gain/mute/NR applied to all of a user's streams - Accessibility: explicit AccessibleName/Description on every control, & mnemonics, Activity log ListBox as durable screen-reader record, AutomationNotification for curated live announcements Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:35:16 +02:00
// Set right after vc_client_create, before vc_connect — lets on_event auto-confirm the
// M4 TOFU gate (VC_EVENT_SERVER_IDENTITY below) for this headless test.
vc_client* client{nullptr};
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
};
static void on_event(void* user, const vc_event* ev) {
auto* s = static_cast<EventStore*>(user);
std::lock_guard lk(s->mu);
switch (ev->type) {
feat(M4): Windows WinForms client, TOFU identity pinning, VAD threshold + always-on mode Core ABI extensions (voicecat.h): - vc_list_channels / vc_list_users / vc_list_user_streams — pull-based snapshot getters for the channel-tree and user-list UI; session_model_mu_ guards cross-thread reads - VC_EVENT_JOIN_RESULT / vc_join_channel — channel join with optional password - VC_EVENT_SERVER_IDENTITY + vc_confirm_server_identity — TOFU gate that blocks io_thread_ until the UI approves or rejects; pins TLS leaf-cert SHA-256 (not declared Ed25519) - vc_get_server_identity_display — Ed25519 fingerprint for human-readable display only - VC_INPUT_ALWAYS_ON = 2 in vc_input_mode — transmit unconditionally, no VAD gate - vc_set_vad_threshold — live RMS threshold update (0.0–1.0); EnergyVadProcessor stores it atomically so the audio RT path reads without a lock C++ implementation: - SessionModel::apply_snapshot / apply_channel_event fixed to populate parent_id, password_protected, and max_users (were permanently zeroed) - TlsContext::peer_cert_fingerprint — SHA-256 of peer leaf cert DER via mbedTLS - TofuStore split into peek (read-only) + pin (write) so first-connect only persists after user approval; tofu_store_path in vc_config for per-user pin file location - TcpAcceptor uses dual-stack IPv6+IPv4 fallback (fixes localhost → ::1 on Windows) - windows-client CMake preset: Release shared DLL, static MinGW runtime, no tools/tests - New C++ tests: test_channel_user_list_abi, test_tofu_flow (14/14 green) Windows client (clients/windows/ — .NET 10 WinForms): - VoiceCat.Interop: LibraryImport P/Invoke surface, UnmanagedCallersOnly callbacks, Channel<VoiceCatEvent> event delivery drained by 30ms WinForms Timer - VoiceCat.App: ConnectDialog (saved servers, DPAPI password storage), ServerIdentity- Dialog (TOFU first-connect / mismatch warning), MainForm (channel TreeView, user ListBox, RichTextBox chat, voice controls, device pickers, VAD/PTT/always-on mode, per-user gain/mute/NR tuning, VAD sensitivity TrackBar, level meter ProgressBar) - PttKeyCaptureDialog — focus-scoped PTT key capture (documented limitation) - PerUserTuningDialog — real-time gain/mute/NR applied to all of a user's streams - Accessibility: explicit AccessibleName/Description on every control, & mnemonics, Activity log ListBox as durable screen-reader record, AutomationNotification for curated live announcements Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:35:16 +02:00
case VC_EVENT_SERVER_IDENTITY:
// No human to ask in a headless test — trust on first connect unconditionally.
vc_confirm_server_identity(s->client, 1);
break;
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
case VC_EVENT_AUTH_RESULT:
s->auth_ok = (ev->result == VC_OK);
s->self_user_id = ev->user_id;
break;
case VC_EVENT_CHANNEL_LIST:
s->channel_list_received = true;
break;
case VC_EVENT_STREAM_STARTED:
s->saw_stream_started = true;
break;
case VC_EVENT_TALK_STATE:
s->talk_events.push_back({ev->user_id, ev->stream_id, ev->u32a != 0});
break;
case VC_EVENT_DISCONNECTED:
s->disconnected = true;
break;
default:
break;
}
s->cv.notify_all();
}
template <typename Pred>
static bool wait_for(EventStore& s, Pred pred, int timeout_ms) {
auto deadline = std::chrono::steady_clock::now() + std::chrono::milliseconds(timeout_ms);
std::unique_lock lk(s.mu);
return s.cv.wait_until(lk, deadline, [&] { return pred(s); });
}
static std::vector<int16_t> make_sine_frame(int frame_idx, float freq_hz,
int frame_samples = 960) {
std::vector<int16_t> pcm(frame_samples);
for (int i = 0; i < frame_samples; ++i) {
float t = static_cast<float>(frame_idx * frame_samples + i) / 48000.0f;
pcm[i] = static_cast<int16_t>(std::sin(2.0f * 3.14159265f * freq_hz * t) * 16000.0f);
}
return pcm;
}
static std::vector<int16_t> make_silence_frame(int frame_samples = 960) {
return std::vector<int16_t>(frame_samples, 0);
}
// Did `talking==true` ever fire for (user_id, stream_id) at index >= `from`?
static bool saw_talking_true(EventStore& s, uint32_t user_id, uint32_t stream_id, size_t from) {
std::lock_guard lk(s.mu);
for (size_t i = from; i < s.talk_events.size(); ++i) {
auto& e = s.talk_events[i];
if (e.user_id == user_id && e.stream_id == stream_id && e.talking) return true;
}
return false;
}
static size_t talk_event_count(EventStore& s) {
std::lock_guard lk(s.mu);
return s.talk_events.size();
}
// ── Test harness ──────────────────────────────────────────────────────────────
static int g_failures = 0;
#define CHECK(cond) \
do { \
if (!(cond)) { \
std::printf("FAIL: %s (%s:%d)\n", #cond, __FILE__, __LINE__); \
++g_failures; \
} \
} while (0)
// ── 1. Device enumeration (no server needed) ────────────────────────────────────
static void test_device_enumeration() {
vc_config cfg{"test-devices", "0.1", VC_LOG_OFF};
vc_callbacks cb{};
vc_client* c = vc_client_create(&cfg, cb);
CHECK(c != nullptr);
for (vc_device_kind kind : {VC_DEVICE_INPUT, VC_DEVICE_OUTPUT}) {
vc_device_list dl{};
vc_result r = vc_list_devices(c, kind, &dl);
#ifdef VOICECAT_HAS_AUDIO
CHECK(r == VC_OK);
// Headless CI build agents may legitimately report zero devices — never assert
// count > 0, only that the call itself succeeded and the list is well-formed.
for (size_t i = 0; i < dl.count; ++i) {
CHECK(dl.items[i].id != nullptr);
CHECK(dl.items[i].name != nullptr);
}
#else
CHECK(r == VC_ERR_NOT_IMPLEMENTED);
#endif
vc_free_device_list(&dl);
vc_free_device_list(&dl); // idempotent — must not crash on a second call
}
vc_client_destroy(c);
std::printf("test_device_enumeration: ok\n");
}
// ── 4. Stereo playback mixer (white-box, no audio hardware needed) ──────────────
#if defined(VOICECAT_HAS_AUDIO) && defined(VOICECAT_HAS_OPUS)
static void test_stereo_mix() {
voicecat::audio::AudioEngine engine;
voicecat::audio::AudioParams p;
p.sample_rate = 48000;
p.capture_channels = 1;
p.playback_channels = 2;
p.frame_ms = 20;
CHECK(engine.start(p)); // capture_cb intentionally omitted — not exercised here
voicecat::codec::OpusParams stereo_params;
stereo_params.stereo = true;
int frame_samples = voicecat::codec::opus_frame_samples(stereo_params);
voicecat::codec::OpusEncoder enc;
CHECK(enc.init(stereo_params));
// Loud left channel, silent right channel — a real downmix would average them into a
// single audible-but-quieter centered sample; true stereo should keep them distinct.
std::vector<int16_t> interleaved(static_cast<size_t>(frame_samples) * 2);
for (int i = 0; i < frame_samples; ++i) {
float t = static_cast<float>(i) / 48000.0f;
interleaved[i * 2] = static_cast<int16_t>(std::sin(2.0f * 3.14159265f * 440.0f * t) * 20000.0f);
interleaved[i * 2 + 1] = 0;
}
uint8_t opus_buf[1500];
int opus_len = enc.encode(interleaved.data(), frame_samples, opus_buf, sizeof(opus_buf));
CHECK(opus_len > 0);
feat: external PCM feed/tap API (vc_stream_feed_pcm + vc_set_pcm_sink) Promotes vc_test_inject_capture (mono-only, TEST-ONLY) to a public, stereo-capable production API and adds a symmetric PCM tap on the receive side. Enables ReplayKit (iOS), ScreenCaptureKit (macOS), bots, soundboards, and custom clients — all without a hardware audio device. Core C++: - voicecat.h: new vc_stream_feed_pcm, vc_pcm_sink_cb typedef, vc_set_pcm_sink; vc_test_inject_capture kept as deprecated alias - audio_engine: stereo-aware inject_capture (channels param + ring reset on channel-count change); atomic pcm_sink_ fired per decoded frame in on_playback; RemoteStream carries user_id/stream_id for RT-safe sink metadata; init_recv_stream takes user_id+stream_id - client.cpp: stream_feed_pcm / set_pcm_sink implementations; sync_remote_streams passes user_id/stream_id to init_recv_stream - voicecat.cpp: trampolines + channels=1/2 validation Tests: test_external_pcm (headless, 3 sub-tests: mono round-trip, stereo feed L≠R, sink metadata+disable). ctest 23/23. Swift: feedPcm / setPcmSink in VoiceCatClient.swift + 4 XCTest smoke tests (ExternalPcmTests.swift). C#: StreamFeedPcm / SetPcmSink in VoiceCatClient.cs + NativeMethods.cs (vc_stream_feed_pcm unsafe P/Invoke, VcPcmSinkCallback delegate, vc_set_pcm_sink via nint) + 4 xUnit smoke tests (ExternalPcmTests.cs). Docs: architecture.md §4 new subsection, voice.md §9 updated (macOS/iOS now reference vc_stream_feed_pcm), protocol.md §8 explicit no-protocol-change note, roadmap.md M5 entry. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-20 17:52:09 +02:00
engine.init_recv_stream(/*ssrc=*/1, stereo_params, /*user_id=*/0, /*stream_id=*/0);
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
voicecat::audio::JitterBuffer::Frame f;
f.seq = 0;
f.timestamp = 0;
f.fec_present = false;
f.payload.assign(opus_buf, opus_buf + opus_len);
engine.push_recv_frame(1, std::move(f));
std::vector<int16_t> out(static_cast<size_t>(frame_samples) * 2, 0);
engine.mix_for_test(out.data(), static_cast<uint32_t>(frame_samples));
// If the engine downmixed (old M3 behavior), every L/R pair would be identical (the
// average of a loud sample and 0). True stereo should show a clear, consistent L != R
// difference across the frame.
int64_t total_diff = 0;
for (int i = 0; i < frame_samples; ++i)
total_diff += std::abs(static_cast<int>(out[i * 2]) - static_cast<int>(out[i * 2 + 1]));
CHECK(total_diff > static_cast<int64_t>(frame_samples) * 1000); // well above decode noise
engine.remove_stream(1);
engine.stop();
std::printf("test_stereo_mix: ok (total_diff=%lld)\n", static_cast<long long>(total_diff));
}
// ── 4a-2. Stereo screen-audio loopback capture (white-box, no audio hardware needed) ──
// Regression for the mono-loopback bug: start_loopback_capture used to hardcode channels=1,
// downmixing the system's stereo mix to mono before the encoder ever saw it (and on_capture_frame
// then upmixed L=R to produce a fake-stereo bitstream). Now the loopback device opens in the
// channel's mode (stereo when the channel is stereo), so the encoder receives real interleaved
// L/R PCM and encodes it directly. This test drives feed_loopback_for_test with a loud-L /
// silent-R stereo signal, encodes it (as on_capture_frame now does for channels==2), decodes,
// and mixes — asserting L != R across the frame. A mono-downmixed-then-upmixed bitstream would
// have L == R. Mirrors test_stereo_mix but routes the encode side through the loopback
// accumulator path that the fix touches (feed_loopback_for_test → on_loopback's accumulator).
#if defined(VOICECAT_HAS_LOOPBACK) && defined(VOICECAT_HAS_OPUS)
static void test_loopback_stereo_capture() {
voicecat::audio::AudioEngine engine;
voicecat::audio::AudioParams p;
p.sample_rate = 48000;
p.capture_channels = 1; // mic path — irrelevant here; loopback has its own channel count
p.playback_channels = 2; // stereo mix output (for mix_for_test below)
p.frame_ms = 20;
CHECK(engine.start(p)); // no capture_cb — the real mic (if any) won't touch capture_accum_
voicecat::codec::OpusParams stereo_params;
stereo_params.stereo = true;
stereo_params.application = voicecat::codec::OpusApplication::Audio; // screen-audio channel
stereo_params.bitrate_bps = 128000; // music/screen-audio channel default
int frame_samples = voicecat::codec::opus_frame_samples(stereo_params);
voicecat::codec::OpusEncoder enc;
CHECK(enc.init(stereo_params));
// Loud left channel, silent right — a real mono downmix would average them into a single
// audible-but-quieter centered sample; true stereo keeps them distinct.
std::vector<int16_t> interleaved(static_cast<size_t>(frame_samples) * 2);
for (int i = 0; i < frame_samples; ++i) {
float t = static_cast<float>(i) / 48000.0f;
interleaved[i * 2] = static_cast<int16_t>(std::sin(2.0f * 3.14159265f * 440.0f * t) * 20000.0f);
interleaved[i * 2 + 1] = 0;
}
// Encode via the loopback accumulator path: feed_loopback_for_test drives on_loopback's
// accumulator and invokes the callback with channels=2 (the fix). The callback encodes
// exactly as on_capture_frame does for real-stereo SCREEN_AUDIO PCM — no upmix.
uint8_t opus_buf[1500];
int opus_len = 0;
int seen_channels = 0;
auto cb = [&](int /*kind*/, const int16_t* pcm, int /*samples*/, int channels) {
seen_channels = channels;
if (channels == 2) {
// The loopback accumulator must have preserved L/R distinctness pre-encode.
int64_t pre_diff = 0;
for (int i = 0; i < frame_samples; ++i)
pre_diff += std::abs(static_cast<int>(pcm[i * 2]) - static_cast<int>(pcm[i * 2 + 1]));
CHECK(pre_diff > static_cast<int64_t>(frame_samples) * 1000);
}
opus_len = enc.encode(pcm, frame_samples, opus_buf, sizeof(opus_buf));
};
engine.feed_loopback_for_test(interleaved.data(), frame_samples, 2, cb);
CHECK(seen_channels == 2); // the loopback path reported stereo, not downmixed mono
CHECK(opus_len > 0);
// Decode + mix — same recv path as test_stereo_mix. A real stereo bitstream should
// survive with L != R; a mono-downmixed-then-upmixed bitstream would have L == R.
feat: external PCM feed/tap API (vc_stream_feed_pcm + vc_set_pcm_sink) Promotes vc_test_inject_capture (mono-only, TEST-ONLY) to a public, stereo-capable production API and adds a symmetric PCM tap on the receive side. Enables ReplayKit (iOS), ScreenCaptureKit (macOS), bots, soundboards, and custom clients — all without a hardware audio device. Core C++: - voicecat.h: new vc_stream_feed_pcm, vc_pcm_sink_cb typedef, vc_set_pcm_sink; vc_test_inject_capture kept as deprecated alias - audio_engine: stereo-aware inject_capture (channels param + ring reset on channel-count change); atomic pcm_sink_ fired per decoded frame in on_playback; RemoteStream carries user_id/stream_id for RT-safe sink metadata; init_recv_stream takes user_id+stream_id - client.cpp: stream_feed_pcm / set_pcm_sink implementations; sync_remote_streams passes user_id/stream_id to init_recv_stream - voicecat.cpp: trampolines + channels=1/2 validation Tests: test_external_pcm (headless, 3 sub-tests: mono round-trip, stereo feed L≠R, sink metadata+disable). ctest 23/23. Swift: feedPcm / setPcmSink in VoiceCatClient.swift + 4 XCTest smoke tests (ExternalPcmTests.swift). C#: StreamFeedPcm / SetPcmSink in VoiceCatClient.cs + NativeMethods.cs (vc_stream_feed_pcm unsafe P/Invoke, VcPcmSinkCallback delegate, vc_set_pcm_sink via nint) + 4 xUnit smoke tests (ExternalPcmTests.cs). Docs: architecture.md §4 new subsection, voice.md §9 updated (macOS/iOS now reference vc_stream_feed_pcm), protocol.md §8 explicit no-protocol-change note, roadmap.md M5 entry. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-20 17:52:09 +02:00
engine.init_recv_stream(/*ssrc=*/3, stereo_params, /*user_id=*/0, /*stream_id=*/0);
voicecat::audio::JitterBuffer::Frame f;
f.seq = 0;
f.timestamp = 0;
f.fec_present = false;
f.payload.assign(opus_buf, opus_buf + opus_len);
engine.push_recv_frame(3, std::move(f));
std::vector<int16_t> out(static_cast<size_t>(frame_samples) * 2, 0);
engine.mix_for_test(out.data(), static_cast<uint32_t>(frame_samples));
int64_t total_diff = 0;
for (int i = 0; i < frame_samples; ++i)
total_diff += std::abs(static_cast<int>(out[i * 2]) - static_cast<int>(out[i * 2 + 1]));
CHECK(total_diff > static_cast<int64_t>(frame_samples) * 1000);
engine.remove_stream(3);
engine.stop();
std::printf("test_loopback_stereo_capture: ok (total_diff=%lld, seen_channels=%d)\n",
static_cast<long long>(total_diff), seen_channels);
}
#endif
// ── 4b. Playout-clock re-sync after a late join / silence gap ────────────────────
// Regression for the "talk indicator lit, no audio" bug: the playout clock free-runs (it
// advances every callback via PLC), while the sender's frame timestamps only advance while it
// is actually transmitting. After a silence gap or a late join the clock drifts past the jitter
// buffer's 500 ms late-drop window, so every real frame is dropped-as-late and the stream is
// permanently silent. on_playback must re-seed the clock to the earliest buffered frame.
static void test_playout_resync() {
voicecat::audio::AudioEngine engine;
voicecat::audio::AudioParams p;
p.sample_rate = 48000;
p.capture_channels = 1;
p.playback_channels = 2;
p.frame_ms = 20;
CHECK(engine.start(p));
voicecat::codec::OpusParams mono_params; // mono = the mic path
mono_params.stereo = false;
int frame_samples = voicecat::codec::opus_frame_samples(mono_params);
voicecat::codec::OpusEncoder enc;
CHECK(enc.init(mono_params));
std::vector<int16_t> sine(static_cast<size_t>(frame_samples));
for (int i = 0; i < frame_samples; ++i) {
float t = static_cast<float>(i) / 48000.0f;
sine[i] = static_cast<int16_t>(std::sin(2.0f * 3.14159265f * 440.0f * t) * 20000.0f);
}
uint8_t opus_buf[1500];
int opus_len = enc.encode(sine.data(), frame_samples, opus_buf, sizeof(opus_buf));
CHECK(opus_len > 0);
feat: external PCM feed/tap API (vc_stream_feed_pcm + vc_set_pcm_sink) Promotes vc_test_inject_capture (mono-only, TEST-ONLY) to a public, stereo-capable production API and adds a symmetric PCM tap on the receive side. Enables ReplayKit (iOS), ScreenCaptureKit (macOS), bots, soundboards, and custom clients — all without a hardware audio device. Core C++: - voicecat.h: new vc_stream_feed_pcm, vc_pcm_sink_cb typedef, vc_set_pcm_sink; vc_test_inject_capture kept as deprecated alias - audio_engine: stereo-aware inject_capture (channels param + ring reset on channel-count change); atomic pcm_sink_ fired per decoded frame in on_playback; RemoteStream carries user_id/stream_id for RT-safe sink metadata; init_recv_stream takes user_id+stream_id - client.cpp: stream_feed_pcm / set_pcm_sink implementations; sync_remote_streams passes user_id/stream_id to init_recv_stream - voicecat.cpp: trampolines + channels=1/2 validation Tests: test_external_pcm (headless, 3 sub-tests: mono round-trip, stereo feed L≠R, sink metadata+disable). ctest 23/23. Swift: feedPcm / setPcmSink in VoiceCatClient.swift + 4 XCTest smoke tests (ExternalPcmTests.swift). C#: StreamFeedPcm / SetPcmSink in VoiceCatClient.cs + NativeMethods.cs (vc_stream_feed_pcm unsafe P/Invoke, VcPcmSinkCallback delegate, vc_set_pcm_sink via nint) + 4 xUnit smoke tests (ExternalPcmTests.cs). Docs: architecture.md §4 new subsection, voice.md §9 updated (macOS/iOS now reference vc_stream_feed_pcm), protocol.md §8 explicit no-protocol-change note, roadmap.md M5 entry. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-20 17:52:09 +02:00
engine.init_recv_stream(/*ssrc=*/2, mono_params, /*user_id=*/0, /*stream_id=*/0);
std::vector<int16_t> out(static_cast<size_t>(frame_samples) * 2, 0);
// Free-run the playout clock with an empty jitter buffer (PLC every callback) far past the
// 500 ms late-drop window — this is what a silence gap / late join does in the field.
for (int i = 0; i < 100; ++i) // ~100 frames @ 20 ms = ~2 s, well past 500 ms
engine.mix_for_test(out.data(), static_cast<uint32_t>(frame_samples));
// Now a real frame arrives carrying a timestamp far behind the free-run clock. Without the
// re-sync it is dropped-as-late and playback stays silent; with it the clock snaps back and
// the frame is decoded and mixed.
voicecat::audio::JitterBuffer::Frame f;
f.seq = 0;
f.timestamp = 0; // stream-relative start, now far behind the drifted playout clock
f.fec_present = false;
f.payload.assign(opus_buf, opus_buf + opus_len);
engine.push_recv_frame(2, std::move(f));
std::fill(out.begin(), out.end(), 0);
engine.mix_for_test(out.data(), static_cast<uint32_t>(frame_samples));
int64_t energy = 0;
for (int16_t s : out) energy += std::abs(static_cast<int>(s));
CHECK(energy > static_cast<int64_t>(frame_samples) * 1000); // audible, not PLC silence
engine.remove_stream(2);
engine.stop();
std::printf("test_playout_resync: ok (energy=%lld)\n", static_cast<long long>(energy));
}
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
#endif // VOICECAT_HAS_AUDIO && VOICECAT_HAS_OPUS
// ── 5. Capture-frame accumulation (white-box, no audio hardware needed) ──────────
// Regression for the capture-side analogue of the playback ring fix: miniaudio's capture
// callback fires at the hardware period (commonly 480 samples on WASAPI shared mode), while
// opus_encode() requires exactly frame_samples_ (960). Sub-frame chunks must be accumulated;
// the callback must receive exactly 960-sample frames regardless of input chunk size.
#ifdef VOICECAT_HAS_AUDIO
static void test_capture_frame_accumulation() {
voicecat::audio::AudioEngine engine;
voicecat::audio::AudioParams p;
p.sample_rate = 48000;
p.capture_channels = 1;
p.frame_ms = 20; // frame_samples_ = 960
std::atomic<int> call_count{0};
std::atomic<bool> wrong_size{false};
constexpr int kExpected = 960;
// Start WITHOUT a capture callback: the real mic (if any) fires on_capture(), but
// on_capture() returns immediately when capture_cb_ is null, so capture_accum_ is
// never touched by the hardware thread. feed_capture_for_test() bypasses capture_cb_
// and drives the same accumulator directly with the explicit `cb` below — no races.
CHECK(engine.start(p));
auto cb = [&](int /*kind*/, const int16_t* /*pcm*/, int samples, int /*channels*/) {
++call_count;
if (samples != kExpected) wrong_size.store(true);
};
// 480-sample (10 ms) input — WASAPI's common hardware period on modern Windows.
// Two 480-chunk inputs → exactly one callback at 960.
std::vector<int16_t> h(480, 1000);
engine.feed_capture_for_test(h.data(), 480, cb);
CHECK(call_count.load() == 0); // half a frame — no callback yet
engine.feed_capture_for_test(h.data(), 480, cb);
CHECK(call_count.load() == 1); // one full frame — callback fired once
// Mis-aligned split: 240 then 720 → still exactly one callback.
std::vector<int16_t> s(240, 500), l(720, 500);
engine.feed_capture_for_test(s.data(), 240, cb);
CHECK(call_count.load() == 1);
engine.feed_capture_for_test(l.data(), 720, cb);
CHECK(call_count.load() == 2);
// 1920-sample input (two Opus frames) → exactly two callbacks.
std::vector<int16_t> d(1920, 800);
engine.feed_capture_for_test(d.data(), 1920, cb);
CHECK(call_count.load() == 4);
CHECK(!wrong_size.load());
engine.stop();
std::printf("test_capture_frame_accumulation: ok (callbacks=%d)\n", call_count.load());
}
#endif // VOICECAT_HAS_AUDIO
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
// ── 2/3. VAD + PTT gate, through the real ABI against a real server ─────────────
static void test_vad_and_ptt_gate() {
auto tmp = std::filesystem::temp_directory_path() /
("vctest_vadptt_" + std::to_string(
std::chrono::steady_clock::now().time_since_epoch().count()));
std::filesystem::create_directories(tmp);
std::string data_dir = tmp.string();
std::atomic<uint16_t> bound_port{0};
std::mutex ready_mu;
std::condition_variable ready_cv;
bool ready{false};
voicecat::server::Config cfg;
cfg.data_dir = data_dir;
cfg.bind_port = 0;
cfg.media_port = 0;
cfg.server_name = "VoiceCat-VadPttTest";
cfg.allow_guests = true;
cfg.on_ready = [&](uint16_t p) {
bound_port.store(p);
{ std::lock_guard lk(ready_mu); ready = true; }
ready_cv.notify_all();
};
voicecat::server::Server server(cfg);
std::thread server_thread([&] { server.run(); });
{
std::unique_lock lk(ready_mu);
bool ok = ready_cv.wait_for(lk, std::chrono::seconds(10), [&] { return ready; });
if (!ok) {
std::printf("FAIL: server did not become ready within 10s\n");
++g_failures;
server.stop();
server_thread.join();
std::filesystem::remove_all(tmp);
return;
}
}
uint16_t port = bound_port.load();
std::printf("test_vad_and_ptt_gate: server ready on :%u\n", port);
EventStore evA;
evA.label = "A";
vc_callbacks cbA{on_event, nullptr, &evA};
vc_config cfgA{"test-A", "0.1", VC_LOG_OFF};
vc_client* clientA = vc_client_create(&cfgA, cbA);
CHECK(clientA != nullptr);
feat(M4): Windows WinForms client, TOFU identity pinning, VAD threshold + always-on mode Core ABI extensions (voicecat.h): - vc_list_channels / vc_list_users / vc_list_user_streams — pull-based snapshot getters for the channel-tree and user-list UI; session_model_mu_ guards cross-thread reads - VC_EVENT_JOIN_RESULT / vc_join_channel — channel join with optional password - VC_EVENT_SERVER_IDENTITY + vc_confirm_server_identity — TOFU gate that blocks io_thread_ until the UI approves or rejects; pins TLS leaf-cert SHA-256 (not declared Ed25519) - vc_get_server_identity_display — Ed25519 fingerprint for human-readable display only - VC_INPUT_ALWAYS_ON = 2 in vc_input_mode — transmit unconditionally, no VAD gate - vc_set_vad_threshold — live RMS threshold update (0.0–1.0); EnergyVadProcessor stores it atomically so the audio RT path reads without a lock C++ implementation: - SessionModel::apply_snapshot / apply_channel_event fixed to populate parent_id, password_protected, and max_users (were permanently zeroed) - TlsContext::peer_cert_fingerprint — SHA-256 of peer leaf cert DER via mbedTLS - TofuStore split into peek (read-only) + pin (write) so first-connect only persists after user approval; tofu_store_path in vc_config for per-user pin file location - TcpAcceptor uses dual-stack IPv6+IPv4 fallback (fixes localhost → ::1 on Windows) - windows-client CMake preset: Release shared DLL, static MinGW runtime, no tools/tests - New C++ tests: test_channel_user_list_abi, test_tofu_flow (14/14 green) Windows client (clients/windows/ — .NET 10 WinForms): - VoiceCat.Interop: LibraryImport P/Invoke surface, UnmanagedCallersOnly callbacks, Channel<VoiceCatEvent> event delivery drained by 30ms WinForms Timer - VoiceCat.App: ConnectDialog (saved servers, DPAPI password storage), ServerIdentity- Dialog (TOFU first-connect / mismatch warning), MainForm (channel TreeView, user ListBox, RichTextBox chat, voice controls, device pickers, VAD/PTT/always-on mode, per-user gain/mute/NR tuning, VAD sensitivity TrackBar, level meter ProgressBar) - PttKeyCaptureDialog — focus-scoped PTT key capture (documented limitation) - PerUserTuningDialog — real-time gain/mute/NR applied to all of a user's streams - Accessibility: explicit AccessibleName/Description on every control, & mnemonics, Activity log ListBox as durable screen-reader record, AutomationNotification for curated live announcements Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:35:16 +02:00
evA.client = clientA;
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
CHECK(vc_connect(clientA, "127.0.0.1", port) == VC_OK);
CHECK(vc_authenticate_guest(clientA, "VP-A") == VC_OK);
CHECK(wait_for(evA, [](EventStore& s) { return s.auth_ok; }, 8000));
CHECK(wait_for(evA, [](EventStore& s) { return s.channel_list_received; }, 3000));
EventStore evB;
evB.label = "B";
vc_callbacks cbB{on_event, nullptr, &evB};
vc_config cfgB{"test-B", "0.1", VC_LOG_OFF};
vc_client* clientB = vc_client_create(&cfgB, cbB);
CHECK(clientB != nullptr);
feat(M4): Windows WinForms client, TOFU identity pinning, VAD threshold + always-on mode Core ABI extensions (voicecat.h): - vc_list_channels / vc_list_users / vc_list_user_streams — pull-based snapshot getters for the channel-tree and user-list UI; session_model_mu_ guards cross-thread reads - VC_EVENT_JOIN_RESULT / vc_join_channel — channel join with optional password - VC_EVENT_SERVER_IDENTITY + vc_confirm_server_identity — TOFU gate that blocks io_thread_ until the UI approves or rejects; pins TLS leaf-cert SHA-256 (not declared Ed25519) - vc_get_server_identity_display — Ed25519 fingerprint for human-readable display only - VC_INPUT_ALWAYS_ON = 2 in vc_input_mode — transmit unconditionally, no VAD gate - vc_set_vad_threshold — live RMS threshold update (0.0–1.0); EnergyVadProcessor stores it atomically so the audio RT path reads without a lock C++ implementation: - SessionModel::apply_snapshot / apply_channel_event fixed to populate parent_id, password_protected, and max_users (were permanently zeroed) - TlsContext::peer_cert_fingerprint — SHA-256 of peer leaf cert DER via mbedTLS - TofuStore split into peek (read-only) + pin (write) so first-connect only persists after user approval; tofu_store_path in vc_config for per-user pin file location - TcpAcceptor uses dual-stack IPv6+IPv4 fallback (fixes localhost → ::1 on Windows) - windows-client CMake preset: Release shared DLL, static MinGW runtime, no tools/tests - New C++ tests: test_channel_user_list_abi, test_tofu_flow (14/14 green) Windows client (clients/windows/ — .NET 10 WinForms): - VoiceCat.Interop: LibraryImport P/Invoke surface, UnmanagedCallersOnly callbacks, Channel<VoiceCatEvent> event delivery drained by 30ms WinForms Timer - VoiceCat.App: ConnectDialog (saved servers, DPAPI password storage), ServerIdentity- Dialog (TOFU first-connect / mismatch warning), MainForm (channel TreeView, user ListBox, RichTextBox chat, voice controls, device pickers, VAD/PTT/always-on mode, per-user gain/mute/NR tuning, VAD sensitivity TrackBar, level meter ProgressBar) - PttKeyCaptureDialog — focus-scoped PTT key capture (documented limitation) - PerUserTuningDialog — real-time gain/mute/NR applied to all of a user's streams - Accessibility: explicit AccessibleName/Description on every control, & mnemonics, Activity log ListBox as durable screen-reader record, AutomationNotification for curated live announcements Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:35:16 +02:00
evB.client = clientB;
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
CHECK(vc_connect(clientB, "127.0.0.1", port) == VC_OK);
CHECK(vc_authenticate_guest(clientB, "VP-B") == VC_OK);
CHECK(wait_for(evB, [](EventStore& s) { return s.auth_ok; }, 8000));
CHECK(wait_for(evB, [](EventStore& s) { return s.channel_list_received; }, 3000));
uint32_t a_uid = 0;
{ std::lock_guard lk(evA.mu); a_uid = evA.self_user_id; }
std::this_thread::sleep_for(std::chrono::milliseconds(500));
vc_stream_desc mic_desc{};
mic_desc.kind = VC_STREAM_MIC;
mic_desc.label = "mic";
uint32_t mic_sid = 0;
CHECK(vc_stream_start(clientA, &mic_desc, &mic_sid) == VC_OK);
CHECK(wait_for(evB, [](EventStore& s) { return s.saw_stream_started; }, 5000));
CHECK(wait_for(evA, [](EventStore& s) { return s.saw_stream_started; }, 5000));
// ── 2a. VAD mode (default), silent PCM: must NOT reach B as a talking edge ──
CHECK(vc_set_input_mode(clientA, VC_INPUT_VOICE_ACTIVATION) == VC_OK);
for (int i = 0; i < 15; ++i) {
auto silence = make_silence_frame();
CHECK(vc_test_inject_capture(clientA, mic_sid, silence.data(), silence.size()) == VC_OK);
std::this_thread::sleep_for(std::chrono::milliseconds(20));
}
CHECK(!saw_talking_true(evB, a_uid, mic_sid, 0));
// ── 2b. VAD mode, loud PCM: must reach B as a talking edge ──────────────────
size_t mark = talk_event_count(evB);
for (int i = 0; i < 20; ++i) {
auto loud = make_sine_frame(i, 440.0f);
CHECK(vc_test_inject_capture(clientA, mic_sid, loud.data(), loud.size()) == VC_OK);
std::this_thread::sleep_for(std::chrono::milliseconds(20));
}
CHECK(wait_for(evB, [&](EventStore& s) {
for (size_t i = mark; i < s.talk_events.size(); ++i) {
auto& e = s.talk_events[i];
if (e.user_id == a_uid && e.stream_id == mic_sid && e.talking) return true;
}
return false;
}, 3000));
// ── 3a. PTT mode, key up: loud PCM must NOT reach B as a new talking edge ───
CHECK(vc_set_input_mode(clientA, VC_INPUT_PUSH_TO_TALK) == VC_OK);
CHECK(vc_set_push_to_talk(clientA, 0) == VC_OK);
// Let any in-flight VAD-driven talking state lapse (hang-time ~300ms) before measuring.
std::this_thread::sleep_for(std::chrono::milliseconds(500));
mark = talk_event_count(evB);
for (int i = 0; i < 20; ++i) {
auto loud = make_sine_frame(i, 440.0f);
CHECK(vc_test_inject_capture(clientA, mic_sid, loud.data(), loud.size()) == VC_OK);
std::this_thread::sleep_for(std::chrono::milliseconds(20));
}
std::this_thread::sleep_for(std::chrono::milliseconds(200));
CHECK(!saw_talking_true(evB, a_uid, mic_sid, mark));
// ── 3b. PTT mode, key down: loud PCM must reach B as a talking edge ─────────
CHECK(vc_set_push_to_talk(clientA, 1) == VC_OK);
mark = talk_event_count(evB);
for (int i = 0; i < 20; ++i) {
auto loud = make_sine_frame(i, 440.0f);
CHECK(vc_test_inject_capture(clientA, mic_sid, loud.data(), loud.size()) == VC_OK);
std::this_thread::sleep_for(std::chrono::milliseconds(20));
}
CHECK(wait_for(evB, [&](EventStore& s) {
for (size_t i = mark; i < s.talk_events.size(); ++i) {
auto& e = s.talk_events[i];
if (e.user_id == a_uid && e.stream_id == mic_sid && e.talking) return true;
}
return false;
}, 3000));
{ std::lock_guard lk(evA.mu); CHECK(!evA.disconnected); }
{ std::lock_guard lk(evB.mu); CHECK(!evB.disconnected); }
vc_disconnect(clientA);
vc_disconnect(clientB);
vc_client_destroy(clientA);
vc_client_destroy(clientB);
server.stop();
server_thread.join();
std::filesystem::remove_all(tmp);
std::printf("test_vad_and_ptt_gate: done\n");
}
feat(ios): audio overhaul, Join/Leave Voice, channel-id sync fix, stereo mic capture Three iOS client problems fixed plus a new core stereo-mic capture ABI: 1. Channel-id sync bug (mic button permanently dimmed): SessionState never synced currentChannelId from the self user's channelId on connect, so the mic button (gated on currentChannelId == 0) stayed dimmed. Added syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522); called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult. Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState. 2. Join/Leave Voice button: replaced icon-only mic toggle with explicit text button (parity with macOS). Mute/deafen disable when not in voice. 3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input port selection, built-in mic orientation/polar patterns, Bluetooth HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay, UserDefaults persistence. AudioSessionManager delegates to it. 4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels() lets the core open the mic device in stereo (2-ch interleaved). LocalStream gains capture_channels; ensure_audio_running reads it; audio_engine.cpp capture_accum_ + on_capture updated to channel-aware accumulation. Test test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper VoiceCatClient.setCaptureChannels. 5. Settings UI rework: AVAudioSession-derived input/output tree replaces miniaudio device picker. 6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj). swift-tools-version 6.0 with swiftLanguageModes .v5. Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md updated; stale 'vc_audio_suspend/resume deferred' claims corrected. Verified: ctest --preset dev 21/21 green; swift test 6/6 green; xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
2026-06-19 13:17:52 +02:00
// ── 5. Stereo mic capture (vc_set_capture_channels) ───────────────────────────
// Verifies that the mic capture accumulator path handles stereo (channels=2) correctly:
// the accumulator is sized to frame_samples_*capture_channels, on_capture forwards the
// correct channel count, and the encoder receives real interleaved L/R PCM (not a mono
// downmix). Mirrors test_loopback_stereo_capture but routes through the mic capture
// accumulator (feed_capture_for_test with channels=2) instead of the loopback path.
// This is the headless CI test for the iOS stereo built-in mic feature (Part D).
#if defined(VOICECAT_HAS_AUDIO) && defined(VOICECAT_HAS_OPUS)
static void test_stereo_mic_capture() {
voicecat::audio::AudioEngine engine;
voicecat::audio::AudioParams p;
p.sample_rate = 48000;
p.capture_channels = 2; // stereo mic capture (vc_set_capture_channels path)
p.playback_channels = 2; // stereo mix output (for mix_for_test below)
p.frame_ms = 20;
CHECK(engine.start(p)); // no capture_cb — the real mic (if any) won't touch capture_accum_
voicecat::codec::OpusParams stereo_params;
stereo_params.stereo = true;
stereo_params.application = voicecat::codec::OpusApplication::Voip; // mic stream
stereo_params.bitrate_bps = 64000; // mic default
int frame_samples = voicecat::codec::opus_frame_samples(stereo_params);
voicecat::codec::OpusEncoder enc;
CHECK(enc.init(stereo_params));
// Loud left channel, silent right — a mono downmix would average them; true stereo
// keeps them distinct (same signal as test_loopback_stereo_capture).
std::vector<int16_t> interleaved(static_cast<size_t>(frame_samples) * 2);
for (int i = 0; i < frame_samples; ++i) {
float t = static_cast<float>(i) / 48000.0f;
interleaved[i * 2] = static_cast<int16_t>(std::sin(2.0f * 3.14159265f * 440.0f * t) * 20000.0f);
interleaved[i * 2 + 1] = 0;
}
// Encode via the mic capture accumulator path: feed_capture_for_test with channels=2
// drives on_capture's accumulator and invokes the callback with channels=2. The callback
// encodes exactly as on_capture_frame does for channels==2 — direct stereo, no upmix.
uint8_t opus_buf[1500];
int opus_len = 0;
int seen_channels = 0;
auto cb = [&](int /*kind*/, const int16_t* pcm, int /*samples*/, int channels) {
seen_channels = channels;
if (channels == 2) {
// The capture accumulator must have preserved L/R distinctness pre-encode.
int64_t pre_diff = 0;
for (int i = 0; i < frame_samples; ++i)
pre_diff += std::abs(static_cast<int>(pcm[i * 2]) - static_cast<int>(pcm[i * 2 + 1]));
CHECK(pre_diff > static_cast<int64_t>(frame_samples) * 1000);
}
opus_len = enc.encode(pcm, frame_samples, opus_buf, sizeof(opus_buf));
};
engine.feed_capture_for_test(interleaved.data(), frame_samples, 2, cb);
CHECK(seen_channels == 2); // the mic capture path reported stereo, not mono
CHECK(opus_len > 0);
// Decode + mix — same recv path as test_stereo_mix. A real stereo bitstream should
// survive with L != R; a mono-downmixed-then-upmixed bitstream would have L == R.
feat: external PCM feed/tap API (vc_stream_feed_pcm + vc_set_pcm_sink) Promotes vc_test_inject_capture (mono-only, TEST-ONLY) to a public, stereo-capable production API and adds a symmetric PCM tap on the receive side. Enables ReplayKit (iOS), ScreenCaptureKit (macOS), bots, soundboards, and custom clients — all without a hardware audio device. Core C++: - voicecat.h: new vc_stream_feed_pcm, vc_pcm_sink_cb typedef, vc_set_pcm_sink; vc_test_inject_capture kept as deprecated alias - audio_engine: stereo-aware inject_capture (channels param + ring reset on channel-count change); atomic pcm_sink_ fired per decoded frame in on_playback; RemoteStream carries user_id/stream_id for RT-safe sink metadata; init_recv_stream takes user_id+stream_id - client.cpp: stream_feed_pcm / set_pcm_sink implementations; sync_remote_streams passes user_id/stream_id to init_recv_stream - voicecat.cpp: trampolines + channels=1/2 validation Tests: test_external_pcm (headless, 3 sub-tests: mono round-trip, stereo feed L≠R, sink metadata+disable). ctest 23/23. Swift: feedPcm / setPcmSink in VoiceCatClient.swift + 4 XCTest smoke tests (ExternalPcmTests.swift). C#: StreamFeedPcm / SetPcmSink in VoiceCatClient.cs + NativeMethods.cs (vc_stream_feed_pcm unsafe P/Invoke, VcPcmSinkCallback delegate, vc_set_pcm_sink via nint) + 4 xUnit smoke tests (ExternalPcmTests.cs). Docs: architecture.md §4 new subsection, voice.md §9 updated (macOS/iOS now reference vc_stream_feed_pcm), protocol.md §8 explicit no-protocol-change note, roadmap.md M5 entry. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-20 17:52:09 +02:00
engine.init_recv_stream(/*ssrc=*/5, stereo_params, /*user_id=*/0, /*stream_id=*/0);
feat(ios): audio overhaul, Join/Leave Voice, channel-id sync fix, stereo mic capture Three iOS client problems fixed plus a new core stereo-mic capture ABI: 1. Channel-id sync bug (mic button permanently dimmed): SessionState never synced currentChannelId from the self user's channelId on connect, so the mic button (gated on currentChannelId == 0) stayed dimmed. Added syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522); called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult. Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState. 2. Join/Leave Voice button: replaced icon-only mic toggle with explicit text button (parity with macOS). Mute/deafen disable when not in voice. 3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input port selection, built-in mic orientation/polar patterns, Bluetooth HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay, UserDefaults persistence. AudioSessionManager delegates to it. 4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels() lets the core open the mic device in stereo (2-ch interleaved). LocalStream gains capture_channels; ensure_audio_running reads it; audio_engine.cpp capture_accum_ + on_capture updated to channel-aware accumulation. Test test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper VoiceCatClient.setCaptureChannels. 5. Settings UI rework: AVAudioSession-derived input/output tree replaces miniaudio device picker. 6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj). swift-tools-version 6.0 with swiftLanguageModes .v5. Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md updated; stale 'vc_audio_suspend/resume deferred' claims corrected. Verified: ctest --preset dev 21/21 green; swift test 6/6 green; xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
2026-06-19 13:17:52 +02:00
voicecat::audio::JitterBuffer::Frame f;
f.seq = 0;
f.timestamp = 0;
f.fec_present = false;
f.payload.assign(opus_buf, opus_buf + opus_len);
engine.push_recv_frame(5, std::move(f));
std::vector<int16_t> out(static_cast<size_t>(frame_samples) * 2, 0);
engine.mix_for_test(out.data(), static_cast<uint32_t>(frame_samples));
int64_t total_diff = 0;
for (int i = 0; i < frame_samples; ++i)
total_diff += std::abs(static_cast<int>(out[i * 2]) - static_cast<int>(out[i * 2 + 1]));
CHECK(total_diff > static_cast<int64_t>(frame_samples) * 1000);
engine.remove_stream(5);
engine.stop();
std::printf("test_stereo_mic_capture: ok (total_diff=%lld, seen_channels=%d)\n",
static_cast<long long>(total_diff), seen_channels);
}
#endif
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
int main() {
test_device_enumeration();
#if defined(VOICECAT_HAS_AUDIO) && defined(VOICECAT_HAS_OPUS)
test_stereo_mix();
#if defined(VOICECAT_HAS_LOOPBACK)
test_loopback_stereo_capture();
#endif
feat(ios): audio overhaul, Join/Leave Voice, channel-id sync fix, stereo mic capture Three iOS client problems fixed plus a new core stereo-mic capture ABI: 1. Channel-id sync bug (mic button permanently dimmed): SessionState never synced currentChannelId from the self user's channelId on connect, so the mic button (gated on currentChannelId == 0) stayed dimmed. Added syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522); called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult. Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState. 2. Join/Leave Voice button: replaced icon-only mic toggle with explicit text button (parity with macOS). Mute/deafen disable when not in voice. 3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input port selection, built-in mic orientation/polar patterns, Bluetooth HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay, UserDefaults persistence. AudioSessionManager delegates to it. 4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels() lets the core open the mic device in stereo (2-ch interleaved). LocalStream gains capture_channels; ensure_audio_running reads it; audio_engine.cpp capture_accum_ + on_capture updated to channel-aware accumulation. Test test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper VoiceCatClient.setCaptureChannels. 5. Settings UI rework: AVAudioSession-derived input/output tree replaces miniaudio device picker. 6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj). swift-tools-version 6.0 with swiftLanguageModes .v5. Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md updated; stale 'vc_audio_suspend/resume deferred' claims corrected. Verified: ctest --preset dev 21/21 green; swift test 6/6 green; xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
2026-06-19 13:17:52 +02:00
test_stereo_mic_capture();
test_playout_resync();
#endif
#ifdef VOICECAT_HAS_AUDIO
test_capture_frame_accumulation();
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
#endif
test_vad_and_ptt_gate();
if (g_failures == 0) {
std::printf("vad_ptt_devices: all checks passed\n");
return 0;
}
std::printf("vad_ptt_devices: %d failure(s)\n", g_failures);
return 1;
}
#else // !VOICECAT_HAS_NET
int main() {
std::printf("vad_ptt_devices: SKIP (VOICECAT_HAS_NET not defined)\n");
return 0;
}
#endif // VOICECAT_HAS_NET