Files
voice-cat/core/src/voicecat.cpp

366 lines
13 KiB
C++
Raw Normal View History

/*
* voicecat.cpp C ABI implementation.
*
* Lifecycle (create/destroy) and trivial accessors are always real. Everything else below
* just delegates to vc_client (core/src/core/client.cpp): under VOICECAT_HAS_NET
* (`dev`/`release`/`server-release` see docs/building.md) that's the real M1M3 implementation;
* under the no-deps `skeleton` preset, client.cpp's `#else` branch returns VC_ERR_NOT_IMPLEMENTED
* for all of it, to keep that skeleton build green.
*/
#include "voicecat.h"
#include <new>
#include "core/client.h"
#define VC_STR2(x) #x
#define VC_STR(x) VC_STR2(x)
extern "C" {
const char* vc_version_string(void) {
static const char* kVersion = VC_STR(VOICECAT_VERSION_MAJOR) "." VC_STR(
VOICECAT_VERSION_MINOR) "." VC_STR(VOICECAT_VERSION_PATCH);
return kVersion;
}
const char* vc_result_string(vc_result code) {
switch (code) {
case VC_OK: return "ok";
case VC_ERR_NOT_IMPLEMENTED: return "not implemented";
case VC_ERR_INVALID_ARG: return "invalid argument";
case VC_ERR_NOT_CONNECTED: return "not connected";
case VC_ERR_ALREADY: return "already in requested state";
case VC_ERR_AUTH_FAILED: return "authentication failed";
case VC_ERR_PERMISSION_DENIED: return "permission denied";
case VC_ERR_TIMEOUT: return "timeout";
case VC_ERR_IO: return "i/o error";
case VC_ERR_PROTOCOL: return "protocol error";
case VC_ERR_CRYPTO: return "crypto error";
case VC_ERR_AUDIO: return "audio error";
case VC_ERR_INTERNAL: return "internal error";
}
return "unknown";
}
vc_client* vc_client_create(const vc_config* cfg, vc_callbacks cb) {
if (cfg == nullptr) return nullptr;
return new (std::nothrow) vc_client(*cfg, cb);
}
void vc_client_destroy(vc_client* c) { delete c; }
/* ── Everything below delegates to vc_client (real or stub, per preset above). ───────── */
vc_result vc_connect(vc_client* c, const char* host, uint16_t port) {
if (c == nullptr || host == nullptr) return VC_ERR_INVALID_ARG;
return c->connect(host, port);
}
vc_result vc_disconnect(vc_client* c) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->disconnect();
}
vc_result vc_authenticate_guest(vc_client* c, const char* nickname) {
if (c == nullptr || nickname == nullptr) return VC_ERR_INVALID_ARG;
return c->authenticate_guest(nickname);
}
vc_result vc_authenticate_user(vc_client* c, const char* username, const char* password) {
if (c == nullptr || username == nullptr || password == nullptr) return VC_ERR_INVALID_ARG;
return c->authenticate_user(username, password);
}
vc_result vc_join_channel(vc_client* c, uint32_t channel_id, const char* password) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->join_channel(channel_id, password);
}
vc_result vc_leave_channel(vc_client* c) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->leave_channel();
}
feat: fix voice join/leave, channel edit defaults, channel-update stream restart Three bugs fixed across the full stack (proto/server/core/ABI/Win/macOS/iOS): 1. Join/Leave Voice now truly subscribes/unsubscribes from the voice plane. Previously the button only toggled the local mic — receiving was always on (gated by channel membership alone). Added a protocol-level voice subscription concept: new SubscribeVoiceRequest/UnsubscribeVoiceRequest/VoiceSubscriptionResult proto messages, User.voice_subscribed field, vc_join_voice/vc_leave_voice C ABI functions, VC_EVENT_VOICE_STATE event, server-side voice_subscribed flag checked by the SFU relay recipient filter, and core-client gating of remote-stream decoder setup. All three clients rewired to subscribe+mic on Join / unsubscribe on Leave. Text chat works regardless of voice subscription. 2. Channel edit dialog now shows the channel's actual current settings. The read struct vc_channel was missing sort_order and audio fields — only the write struct vc_channel_info had them. Extended vc_channel with both (additive, no ABI break), updated the session model and list_channels marshaling to populate them, and updated all three clients' edit callers to use actual channel info instead of hardcoded defaults. 3. Channel parameter updates now automatically restart everyone's streams. Previously editing a channel's audio config persisted and broadcast a ChannelEvent::UPDATED, but no layer restarted streams — encoders/decoders are frozen at announce time. handle_channel_event now detects audio-config changes on the user's current channel and stop->starts each active local stream. The server reads the updated config on re-announce; peers wire up fresh decoders at the new ssrc. All 29 CTest tests pass; Windows DLL + C# client build clean. Apple clients not yet compile-verified (Windows environment).
2026-06-24 14:29:39 +02:00
vc_result vc_join_voice(vc_client* c) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->join_voice();
}
vc_result vc_leave_voice(vc_client* c) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->leave_voice();
}
vc_result vc_stream_start(vc_client* c, const vc_stream_desc* desc, uint32_t* out_stream_id) {
if (c == nullptr || desc == nullptr) return VC_ERR_INVALID_ARG;
return c->stream_start(*desc, out_stream_id);
}
vc_result vc_stream_stop(vc_client* c, uint32_t stream_id) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->stream_stop(stream_id);
}
vc_result vc_set_input_device(vc_client* c, uint32_t stream_id, const char* device_id) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->set_input_device(stream_id, device_id);
}
vc_result vc_set_input_mode(vc_client* c, vc_input_mode mode) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->set_input_mode(mode);
}
feat(M4): Windows WinForms client, TOFU identity pinning, VAD threshold + always-on mode Core ABI extensions (voicecat.h): - vc_list_channels / vc_list_users / vc_list_user_streams — pull-based snapshot getters for the channel-tree and user-list UI; session_model_mu_ guards cross-thread reads - VC_EVENT_JOIN_RESULT / vc_join_channel — channel join with optional password - VC_EVENT_SERVER_IDENTITY + vc_confirm_server_identity — TOFU gate that blocks io_thread_ until the UI approves or rejects; pins TLS leaf-cert SHA-256 (not declared Ed25519) - vc_get_server_identity_display — Ed25519 fingerprint for human-readable display only - VC_INPUT_ALWAYS_ON = 2 in vc_input_mode — transmit unconditionally, no VAD gate - vc_set_vad_threshold — live RMS threshold update (0.0–1.0); EnergyVadProcessor stores it atomically so the audio RT path reads without a lock C++ implementation: - SessionModel::apply_snapshot / apply_channel_event fixed to populate parent_id, password_protected, and max_users (were permanently zeroed) - TlsContext::peer_cert_fingerprint — SHA-256 of peer leaf cert DER via mbedTLS - TofuStore split into peek (read-only) + pin (write) so first-connect only persists after user approval; tofu_store_path in vc_config for per-user pin file location - TcpAcceptor uses dual-stack IPv6+IPv4 fallback (fixes localhost → ::1 on Windows) - windows-client CMake preset: Release shared DLL, static MinGW runtime, no tools/tests - New C++ tests: test_channel_user_list_abi, test_tofu_flow (14/14 green) Windows client (clients/windows/ — .NET 10 WinForms): - VoiceCat.Interop: LibraryImport P/Invoke surface, UnmanagedCallersOnly callbacks, Channel<VoiceCatEvent> event delivery drained by 30ms WinForms Timer - VoiceCat.App: ConnectDialog (saved servers, DPAPI password storage), ServerIdentity- Dialog (TOFU first-connect / mismatch warning), MainForm (channel TreeView, user ListBox, RichTextBox chat, voice controls, device pickers, VAD/PTT/always-on mode, per-user gain/mute/NR tuning, VAD sensitivity TrackBar, level meter ProgressBar) - PttKeyCaptureDialog — focus-scoped PTT key capture (documented limitation) - PerUserTuningDialog — real-time gain/mute/NR applied to all of a user's streams - Accessibility: explicit AccessibleName/Description on every control, & mnemonics, Activity log ListBox as durable screen-reader record, AutomationNotification for curated live announcements Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:35:16 +02:00
vc_result vc_set_vad_threshold(vc_client* c, float threshold) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->set_vad_threshold(threshold);
}
vc_result vc_set_push_to_talk(vc_client* c, int active) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->set_push_to_talk(active != 0);
}
vc_result vc_set_self_mute(vc_client* c, int mic_muted, int deafened) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->set_self_mute(mic_muted != 0, deafened != 0);
}
feat(windows): UI overhaul -- toolbar, unified log, PM windows, channel counts, output volume - Voice actions (Join Voice, Share Screen Audio) moved to a ToolStrip toolbar and a new Voice menu in the menu bar; removed from the bottom voice panel - Activity log and chat log collapsed into a single RichTextBox (rtbLog); activity events appear in gray, chat messages in default color - Private messaging reworked: each conversation opens in its own modeless PrivateMessageForm instead of sharing the main chat log via a scope dropdown; cboScope removed; main compose bar always sends to the current channel - New "Messages -> New Private Message..." menu item (Ctrl+P) opens a UserPickerDialog listing all connected server users (not just the current channel) so you can PM anyone on the server - Channel tree now shows live user counts, e.g. "General (3)" -- counts sourced from the existing _users dictionary which already tracks all server users with channel IDs - Global output volume slider (TrackBar, 0-100, default 80) added to the right panel; wired to new vc_set_output_volume C ABI function that applies a master gain multiplier in the audio engine playback callback after mixing all streams - vc_set_output_volume added end-to-end: voicecat.h, audio_engine.h/.cpp, client.h/.cpp, voicecat.cpp, NativeMethods.cs, VoiceCatClient.cs - Documented Windows PowerShell ctest requirement in AGENTS.md and CLAUDE.md: MinGW binaries exit 0xc0000139 in Git Bash; always run ctest/.exe via PowerShell 22/22 ctest green (PowerShell); dotnet build 0 warnings. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-20 14:24:54 +02:00
vc_result vc_set_output_volume(vc_client* c, float gain) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->set_output_volume(gain);
}
feat(clients): persist input settings, add mic input gain, fix iOS chat + VoiceOver Input mode (VAD/PTT/Always-On), VAD threshold, and the new mic gain were applied to the core + UI but never saved, so every relaunch reset to VAD defaults. Each client now persists them and re-applies on connect: - iOS: UserDefaults (SessionState.loadAndApplyVoiceSettings + setter writes) - macOS: UserDefaults via MainWindowController didSet + loadPersistedAudioSettings (settings window also restores the VAD slider from the stored threshold) - Windows: new Models/VoiceSettings.cs (JSON at %AppData%\VoiceCat\voice.json, mirrors FeedbackSettings) loaded/applied in MainForm Add global send-side mic gain API vc_set_input_gain (applied to MIC PCM in on_capture_frame before the VAD gate, clamped to int16) + Swift/C# bindings, and a 0-300% (default 100%) mic-volume slider on all three clients. Fix iOS chat: ChatView called sendText(scope:.channel) with no targetId (0), so channel messages went nowhere; now passes session.currentChannelId. Fix iOS per-user tuning for VoiceOver: the tuning sheet was long-press .contextMenu only (invisible to VoiceOver); UserRow now also exposes the same buttons via .accessibilityActions (no visual change). Verified: core builds clean; ctest 24/27 (3 pre-existing teardown crashes, reproduced with changes stashed); VoiceCatMac + VoiceCatiOS (arm64 sim) build SUCCEEDED; VoiceCat.Interop dotnet build succeeded. Windows App not built (WinForms can't build on macOS) — follows existing patterns.
2026-06-23 03:35:26 +02:00
vc_result vc_set_input_gain(vc_client* c, float gain) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->set_input_gain(gain);
}
feat(audio): real noise suppression via vendored RNNoise (send + receive) The two-sided NR plumbing (RemoteStream::recv_ns + the per-listener vc_set_remote_stream noise_reduction toggle) was wired but inert: ApmProcessor::create() returned a no-op passthrough, because the originally-planned webrtc-audio-processing has no working Windows/macOS build. Drop in RNNoise as the real backend behind the same ApmProcessor interface, lighting up both NR paths. - Vendor RNNoise (BSD-3 + CC0) at third_party/rnnoise/ — the vcpkg port is !windows !arm, so it can't cover our primary targets. Shrunk int8 model (78MB -> 11.7MB via upstream scripts/shrink_model.sh), built as a standalone C static lib with no RTCD (portable scalar path on x86, auto-NEON on arm64) under -DDISABLE_DEBUG_FLOAT. Model is baked in (rnnoise_create(NULL)); no runtime file. - New RnnoiseProcessor (core/src/audio/apm_processor.cpp) selected by ApmProcessor::create() when VOICECAT_HAS_NS. Mono/48kHz/480-sample; our clock is fixed 48kHz and Opus frame sizes are multiples of 480, so no resampling. RT-safe: allocates at construction, lock-free in the capture/playback callbacks. - Receive-side: lit up via the factory; gated to mono streams (a stereo stream is a screen-audio share, not voice). - Send-side (new): vc_set_input_noise_reduction(client, enable) ABI + vc_client::mic_ns_, run before input gain/VAD in on_capture_frame. A stereo mic is downmixed to mono ONLY when NR is on — with NR off a stereo mic keeps full stereo (never collapse mic quality unasked). - Enable C as a project language for the vendored lib. - New noise_suppression test: white noise through ApmProcessor::create() drops ~99.9% RMS. ctest --preset dev green, 28/28. windows-client DLL builds clean with vc_set_input_noise_reduction exported, system-only deps. - Docs synced: voice.md §10, tech-stack.md §1/§5, third_party/README.md, vcpkg.json note, PROGRESS.md, CLAUDE.md. Client on/off UI toggles (Windows/macOS/iOS) are the remaining follow-up. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-23 13:30:54 +02:00
vc_result vc_set_input_noise_reduction(vc_client* c, int enable) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->set_input_noise_reduction(enable != 0);
}
vc_result vc_set_remote_stream(vc_client* c, uint32_t user_id, uint32_t stream_id, float gain,
int muted, int noise_reduction) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->set_remote_stream(user_id, stream_id, gain, muted != 0, noise_reduction != 0);
}
vc_result vc_get_remote_stream(vc_client* c, uint32_t user_id, uint32_t stream_id,
vc_remote_stream_state* out) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->get_remote_stream(user_id, stream_id, out);
}
feat(M3): multi-stream & per-channel tuning Implements docs/roadmap.md M3: multiple concurrent streams per user (MIC + SCREEN_AUDIO + AUX_DEVICE), independent per-stream receiver gain/mute/noise- reduction, talk indicators, and enforced per-channel Opus configurability (mono/stereo, bitrate, frame size, FEC/DTX, application). Bugs fixed along the way (found while implementing, not pre-existing scope): - Server hard-coded stream_id=1 for every announce, so a second stream from the same user silently overwrote the first in SessionRegistry::set_user_stream. Now a per-session counter (ConnSession::next_stream_id_); handle_stream_stop validates against announced_stream_ids_ before clearing. - Client dropped mode/dtx/complexity/application from effective_audio even for the single M2 stream -- only sample_rate/bitrate_bps/frame_ms/fec were ever applied to OpusParams. Fixed on both the send (handle_stream_announce_result) and receive (sync_remote_streams) paths via a shared opus_params_from_audio_config() helper. - OpusEncoder always used OPUS_APPLICATION_VOIP; added OpusParams::application and wired it through. - on_playback's per-stream decode passed the wrong frame_size to opus_decode (total samples instead of samples-per-channel), which would have overflowed the decode buffer for any stereo stream. - teardown_voice() raced when called concurrently from run_io()'s own cleanup and from disconnect() on a different thread -- both could see udp_thread_/talk_timer_thread_ as joinable() at once and race to join() the same std::thread (intermittent std::system_error under ctest). Fixed with a teardown_mu_ guard instead of carrying the flake forward. New: - Per-channel AudioConfig: SessionRegistry now seeds Lobby (mono/24kbps/VOIP/ FEC+DTX) and a new "Music Room" channel (stereo/128kbps/AUDIO/no DTX); handle_stream_announce enforces the channel's config, clamping (not overriding) bitrate_bps to its ceiling. - core/src/core/client.h/.cpp: local-stream state is now a std::unordered_map<int, LocalStream> keyed by vc_stream_kind, with request_id-correlated announce/result handling (request_id already round-tripped on the wire; just wasn't read before). on_capture_frame is kind-aware and upmixes mono capture to stereo when a stream's config calls for it. set_self_mute's mic_muted now only gates the MIC kind. NS is wired through set_remote_stream. New run_talk_timer() thread emits VC_EVENT_TALK_STATE from both remote and local edge detection. - core/src/audio/audio_engine.h/.cpp: kind-keyed injection taps (inject_capture), stereo-to-mono downmix at the decode/mix boundary, RemoteStream gains recv_ns (lazy ApmProcessor) + noise_reduction_enabled and last_voice_ms/talking; new set_stream_noise_reduction() and poll_talk_transitions(). - core/src/session/session.h/.cpp: Stream now carries the full AudioConfig, not just sample_rate/frame_ms. - New additive C ABI (core/include/voicecat.h): vc_audio_config + vc_get_stream_audio_config (effective Opus config for any stream you own or a peer's); vc_test_inject_capture (test-only synthetic PCM injection, clearly marked, mirrors AudioEngine::inject_capture). - tests/test_m3_multistream.cpp: the M3 exit criterion through the real ABI (mirrors test_voice_client_abi.cpp's approach, not raw sockets) -- two concurrent local streams, independent gain/mute/NS control, per-channel config divergence via vc_get_stream_audio_config, talk indicators. Explicitly out of scope for this pass (tracked in PROGRESS.md, not silently dropped): VAD/PTT input gate + device enumeration; real WASAPI loopback capture for SCREEN_AUDIO (synthetic injection only); true stereo playback output (AudioEngine's mixer/output device stays mono -- Opus itself is fully stereo-correct on the wire). ctest --test-dir build/m1-dev: 11/11 green, verified across 3 consecutive full-suite runs plus 8 standalone runs of the new test. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 14:12:37 +02:00
vc_result vc_get_stream_audio_config(vc_client* c, uint32_t user_id, uint32_t stream_id,
vc_audio_config* out) {
if (c == nullptr || out == nullptr) return VC_ERR_INVALID_ARG;
return c->get_stream_audio_config(user_id, stream_id, out);
}
vc_result vc_test_inject_capture(vc_client* c, uint32_t stream_id, const int16_t* pcm,
size_t samples) {
if (c == nullptr || pcm == nullptr) return VC_ERR_INVALID_ARG;
feat: external PCM feed/tap API (vc_stream_feed_pcm + vc_set_pcm_sink) Promotes vc_test_inject_capture (mono-only, TEST-ONLY) to a public, stereo-capable production API and adds a symmetric PCM tap on the receive side. Enables ReplayKit (iOS), ScreenCaptureKit (macOS), bots, soundboards, and custom clients — all without a hardware audio device. Core C++: - voicecat.h: new vc_stream_feed_pcm, vc_pcm_sink_cb typedef, vc_set_pcm_sink; vc_test_inject_capture kept as deprecated alias - audio_engine: stereo-aware inject_capture (channels param + ring reset on channel-count change); atomic pcm_sink_ fired per decoded frame in on_playback; RemoteStream carries user_id/stream_id for RT-safe sink metadata; init_recv_stream takes user_id+stream_id - client.cpp: stream_feed_pcm / set_pcm_sink implementations; sync_remote_streams passes user_id/stream_id to init_recv_stream - voicecat.cpp: trampolines + channels=1/2 validation Tests: test_external_pcm (headless, 3 sub-tests: mono round-trip, stereo feed L≠R, sink metadata+disable). ctest 23/23. Swift: feedPcm / setPcmSink in VoiceCatClient.swift + 4 XCTest smoke tests (ExternalPcmTests.swift). C#: StreamFeedPcm / SetPcmSink in VoiceCatClient.cs + NativeMethods.cs (vc_stream_feed_pcm unsafe P/Invoke, VcPcmSinkCallback delegate, vc_set_pcm_sink via nint) + 4 xUnit smoke tests (ExternalPcmTests.cs). Docs: architecture.md §4 new subsection, voice.md §9 updated (macOS/iOS now reference vc_stream_feed_pcm), protocol.md §8 explicit no-protocol-change note, roadmap.md M5 entry. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-20 17:52:09 +02:00
return c->stream_feed_pcm(stream_id, pcm, samples, 1);
}
vc_result vc_stream_feed_pcm(vc_client* c, uint32_t stream_id, const int16_t* pcm,
size_t samples_per_channel, uint32_t channels) {
if (c == nullptr || pcm == nullptr) return VC_ERR_INVALID_ARG;
if (channels != 1 && channels != 2) return VC_ERR_INVALID_ARG;
return c->stream_feed_pcm(stream_id, pcm, samples_per_channel, channels);
}
vc_result vc_set_pcm_sink(vc_client* c, vc_pcm_sink_cb cb, void* user) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->set_pcm_sink(cb, user);
feat(M3): multi-stream & per-channel tuning Implements docs/roadmap.md M3: multiple concurrent streams per user (MIC + SCREEN_AUDIO + AUX_DEVICE), independent per-stream receiver gain/mute/noise- reduction, talk indicators, and enforced per-channel Opus configurability (mono/stereo, bitrate, frame size, FEC/DTX, application). Bugs fixed along the way (found while implementing, not pre-existing scope): - Server hard-coded stream_id=1 for every announce, so a second stream from the same user silently overwrote the first in SessionRegistry::set_user_stream. Now a per-session counter (ConnSession::next_stream_id_); handle_stream_stop validates against announced_stream_ids_ before clearing. - Client dropped mode/dtx/complexity/application from effective_audio even for the single M2 stream -- only sample_rate/bitrate_bps/frame_ms/fec were ever applied to OpusParams. Fixed on both the send (handle_stream_announce_result) and receive (sync_remote_streams) paths via a shared opus_params_from_audio_config() helper. - OpusEncoder always used OPUS_APPLICATION_VOIP; added OpusParams::application and wired it through. - on_playback's per-stream decode passed the wrong frame_size to opus_decode (total samples instead of samples-per-channel), which would have overflowed the decode buffer for any stereo stream. - teardown_voice() raced when called concurrently from run_io()'s own cleanup and from disconnect() on a different thread -- both could see udp_thread_/talk_timer_thread_ as joinable() at once and race to join() the same std::thread (intermittent std::system_error under ctest). Fixed with a teardown_mu_ guard instead of carrying the flake forward. New: - Per-channel AudioConfig: SessionRegistry now seeds Lobby (mono/24kbps/VOIP/ FEC+DTX) and a new "Music Room" channel (stereo/128kbps/AUDIO/no DTX); handle_stream_announce enforces the channel's config, clamping (not overriding) bitrate_bps to its ceiling. - core/src/core/client.h/.cpp: local-stream state is now a std::unordered_map<int, LocalStream> keyed by vc_stream_kind, with request_id-correlated announce/result handling (request_id already round-tripped on the wire; just wasn't read before). on_capture_frame is kind-aware and upmixes mono capture to stereo when a stream's config calls for it. set_self_mute's mic_muted now only gates the MIC kind. NS is wired through set_remote_stream. New run_talk_timer() thread emits VC_EVENT_TALK_STATE from both remote and local edge detection. - core/src/audio/audio_engine.h/.cpp: kind-keyed injection taps (inject_capture), stereo-to-mono downmix at the decode/mix boundary, RemoteStream gains recv_ns (lazy ApmProcessor) + noise_reduction_enabled and last_voice_ms/talking; new set_stream_noise_reduction() and poll_talk_transitions(). - core/src/session/session.h/.cpp: Stream now carries the full AudioConfig, not just sample_rate/frame_ms. - New additive C ABI (core/include/voicecat.h): vc_audio_config + vc_get_stream_audio_config (effective Opus config for any stream you own or a peer's); vc_test_inject_capture (test-only synthetic PCM injection, clearly marked, mirrors AudioEngine::inject_capture). - tests/test_m3_multistream.cpp: the M3 exit criterion through the real ABI (mirrors test_voice_client_abi.cpp's approach, not raw sockets) -- two concurrent local streams, independent gain/mute/NS control, per-channel config divergence via vc_get_stream_audio_config, talk indicators. Explicitly out of scope for this pass (tracked in PROGRESS.md, not silently dropped): VAD/PTT input gate + device enumeration; real WASAPI loopback capture for SCREEN_AUDIO (synthetic injection only); true stereo playback output (AudioEngine's mixer/output device stays mono -- Opus itself is fully stereo-correct on the wire). ctest --test-dir build/m1-dev: 11/11 green, verified across 3 consecutive full-suite runs plus 8 standalone runs of the new test. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 14:12:37 +02:00
}
vc_result vc_set_mixed_output_sink(vc_client* c, vc_mixed_output_cb cb, void* user) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->set_mixed_output_sink(cb, user);
}
vc_result vc_set_external_playback(vc_client* c, int enable) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->set_external_playback(enable != 0);
}
feat(ios): audio overhaul, Join/Leave Voice, channel-id sync fix, stereo mic capture Three iOS client problems fixed plus a new core stereo-mic capture ABI: 1. Channel-id sync bug (mic button permanently dimmed): SessionState never synced currentChannelId from the self user's channelId on connect, so the mic button (gated on currentChannelId == 0) stayed dimmed. Added syncSelfChannel() (mirrors macOS MainWindowController.swift:461,491,522); called from init/.channelList/.userJoined/.userLeft/.userUpdated/.joinResult. Added applyServerMuteState() + serverMuted/serverDeafened to VoiceState. 2. Join/Leave Voice button: replaced icon-only mic toggle with explicit text button (parity with macOS). Mute/deafen disable when not in voice. 3. IOSAudioRouter.swift (new): full AVAudioSession routing layer — input port selection, built-in mic orientation/polar patterns, Bluetooth HFP/A2DP/Off modes, Standard/Raw mic processing, stereo capture, AirPlay, UserDefaults persistence. AudioSessionManager delegates to it. 4. Core stereo-mic capture (append-only ABI): vc_set_capture_channels() lets the core open the mic device in stereo (2-ch interleaved). LocalStream gains capture_channels; ensure_audio_running reads it; audio_engine.cpp capture_accum_ + on_capture updated to channel-aware accumulation. Test test_stereo_mic_capture (headless, L!=R stereo round-trip). Swift wrapper VoiceCatClient.setCaptureChannels. 5. Settings UI rework: AVAudioSession-derived input/output tree replaces miniaudio device picker. 6. iOS deployment target raised to 18.0 (Package.swift + project.pbxproj). swift-tools-version 6.0 with swiftLanguageModes .v5. Docs: tech-stack.md, architecture.md, voice.md, roadmap.md, building.md updated; stale 'vc_audio_suspend/resume deferred' claims corrected. Verified: ctest --preset dev 21/21 green; swift test 6/6 green; xcodebuild -target VoiceCatiOS -sdk iphonesimulator BUILD SUCCEEDED.
2026-06-19 13:17:52 +02:00
vc_result vc_set_capture_channels(vc_client* c, uint32_t stream_id, uint32_t channels) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->set_capture_channels(stream_id, channels);
}
vc_result vc_send_text(vc_client* c, vc_text_scope scope, uint32_t target_id,
const char* utf8) {
if (c == nullptr || utf8 == nullptr) return VC_ERR_INVALID_ARG;
return c->send_text(scope, target_id, utf8);
}
vc_result vc_list_devices(vc_client* c, vc_device_kind kind, vc_device_list* out) {
if (c == nullptr || out == nullptr) return VC_ERR_INVALID_ARG;
return c->list_devices(kind, out);
}
void vc_free_device_list(vc_device_list* list) {
feat: device enumeration, VAD/PTT input gate, stereo playback, WASAPI loopback Closes the three items PROGRESS.md's M3 section explicitly carried forward as out of scope: - Device enumeration (vc_list_devices) + input device selection (vc_set_input_device), backed by AudioEngine::enumerate_devices() via miniaudio's ma_context_get_devices. Device ids are opaque hex-encoded ma_device_id strings. - VAD/PTT send-side input gate (vc_set_input_mode, vc_set_push_to_talk). webrtc-audio-processing (the originally-planned APM) has no working Windows/MSVC build upstream (GCC-only Meson, unfinished MinGW support, hard abseil-cpp dependency), so VAD is a new lightweight, dependency-free energy/RMS processor (EnergyVadProcessor) behind the existing ApmProcessor interface. Gating is MIC-only; SCREEN_AUDIO/AUX_DEVICE always bypass it. - True stereo playback: AudioEngine's mixer and output device now carry stereo end-to-end (mono streams upmix L=R) instead of downmixing decoded stereo streams to mono before mixing. - Real WASAPI loopback capture for SCREEN_AUDIO (Windows-only, via miniaudio's loopback device type), replacing test-only injection as the production capture path. Also: vccli gains --list-devices, --input-device, --input-mode, and --share-screen-audio flags, plus a stdin command loop (ptt on/off, mode vad/ptt) for manual verification. New test_vad_ptt_devices.cpp covers all four items (ABI-level + a white-box AudioEngine stereo-mix check). Docs updated to match: voice.md, roadmap.md (decision-log entry superseding the original webrtc-audio-processing choice), tech-stack.md, README.md, architecture.md, CLAUDE.md, PROGRESS.md. Still explicitly out of scope, documented not silently dropped: real webrtc-audio-processing/AEC (no AEC/NS/AGC exists at all yet), macOS/iOS SCREEN_AUDIO capture, process-specific loopback, and a pre-existing RT-thread rule violation in the capture path that predates this work. Verified: ctest 12/12 green across 3 consecutive full-suite runs (both dev and m1-dev presets build clean); test_vad_ptt_devices passed 5 consecutive standalone runs; manually verified live (vccli --list-devices against real hardware, vccli --voice --input-mode vad streaming without incident). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 16:11:52 +02:00
if (list == nullptr || list->items == nullptr) return;
for (size_t i = 0; i < list->count; ++i) {
delete[] list->items[i].id;
delete[] list->items[i].name;
}
delete[] list->items;
list->items = nullptr;
list->count = 0;
}
feat(M4): Windows WinForms client, TOFU identity pinning, VAD threshold + always-on mode Core ABI extensions (voicecat.h): - vc_list_channels / vc_list_users / vc_list_user_streams — pull-based snapshot getters for the channel-tree and user-list UI; session_model_mu_ guards cross-thread reads - VC_EVENT_JOIN_RESULT / vc_join_channel — channel join with optional password - VC_EVENT_SERVER_IDENTITY + vc_confirm_server_identity — TOFU gate that blocks io_thread_ until the UI approves or rejects; pins TLS leaf-cert SHA-256 (not declared Ed25519) - vc_get_server_identity_display — Ed25519 fingerprint for human-readable display only - VC_INPUT_ALWAYS_ON = 2 in vc_input_mode — transmit unconditionally, no VAD gate - vc_set_vad_threshold — live RMS threshold update (0.0–1.0); EnergyVadProcessor stores it atomically so the audio RT path reads without a lock C++ implementation: - SessionModel::apply_snapshot / apply_channel_event fixed to populate parent_id, password_protected, and max_users (were permanently zeroed) - TlsContext::peer_cert_fingerprint — SHA-256 of peer leaf cert DER via mbedTLS - TofuStore split into peek (read-only) + pin (write) so first-connect only persists after user approval; tofu_store_path in vc_config for per-user pin file location - TcpAcceptor uses dual-stack IPv6+IPv4 fallback (fixes localhost → ::1 on Windows) - windows-client CMake preset: Release shared DLL, static MinGW runtime, no tools/tests - New C++ tests: test_channel_user_list_abi, test_tofu_flow (14/14 green) Windows client (clients/windows/ — .NET 10 WinForms): - VoiceCat.Interop: LibraryImport P/Invoke surface, UnmanagedCallersOnly callbacks, Channel<VoiceCatEvent> event delivery drained by 30ms WinForms Timer - VoiceCat.App: ConnectDialog (saved servers, DPAPI password storage), ServerIdentity- Dialog (TOFU first-connect / mismatch warning), MainForm (channel TreeView, user ListBox, RichTextBox chat, voice controls, device pickers, VAD/PTT/always-on mode, per-user gain/mute/NR tuning, VAD sensitivity TrackBar, level meter ProgressBar) - PttKeyCaptureDialog — focus-scoped PTT key capture (documented limitation) - PerUserTuningDialog — real-time gain/mute/NR applied to all of a user's streams - Accessibility: explicit AccessibleName/Description on every control, & mnemonics, Activity log ListBox as durable screen-reader record, AutomationNotification for curated live announcements Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:35:16 +02:00
vc_result vc_list_channels(vc_client* c, vc_channel_list* out) {
if (c == nullptr || out == nullptr) return VC_ERR_INVALID_ARG;
return c->list_channels(out);
}
void vc_free_channel_list(vc_channel_list* list) {
if (list == nullptr || list->items == nullptr) return;
for (size_t i = 0; i < list->count; ++i) {
delete[] list->items[i].name;
delete[] list->items[i].topic;
}
feat(M4): Windows WinForms client, TOFU identity pinning, VAD threshold + always-on mode Core ABI extensions (voicecat.h): - vc_list_channels / vc_list_users / vc_list_user_streams — pull-based snapshot getters for the channel-tree and user-list UI; session_model_mu_ guards cross-thread reads - VC_EVENT_JOIN_RESULT / vc_join_channel — channel join with optional password - VC_EVENT_SERVER_IDENTITY + vc_confirm_server_identity — TOFU gate that blocks io_thread_ until the UI approves or rejects; pins TLS leaf-cert SHA-256 (not declared Ed25519) - vc_get_server_identity_display — Ed25519 fingerprint for human-readable display only - VC_INPUT_ALWAYS_ON = 2 in vc_input_mode — transmit unconditionally, no VAD gate - vc_set_vad_threshold — live RMS threshold update (0.0–1.0); EnergyVadProcessor stores it atomically so the audio RT path reads without a lock C++ implementation: - SessionModel::apply_snapshot / apply_channel_event fixed to populate parent_id, password_protected, and max_users (were permanently zeroed) - TlsContext::peer_cert_fingerprint — SHA-256 of peer leaf cert DER via mbedTLS - TofuStore split into peek (read-only) + pin (write) so first-connect only persists after user approval; tofu_store_path in vc_config for per-user pin file location - TcpAcceptor uses dual-stack IPv6+IPv4 fallback (fixes localhost → ::1 on Windows) - windows-client CMake preset: Release shared DLL, static MinGW runtime, no tools/tests - New C++ tests: test_channel_user_list_abi, test_tofu_flow (14/14 green) Windows client (clients/windows/ — .NET 10 WinForms): - VoiceCat.Interop: LibraryImport P/Invoke surface, UnmanagedCallersOnly callbacks, Channel<VoiceCatEvent> event delivery drained by 30ms WinForms Timer - VoiceCat.App: ConnectDialog (saved servers, DPAPI password storage), ServerIdentity- Dialog (TOFU first-connect / mismatch warning), MainForm (channel TreeView, user ListBox, RichTextBox chat, voice controls, device pickers, VAD/PTT/always-on mode, per-user gain/mute/NR tuning, VAD sensitivity TrackBar, level meter ProgressBar) - PttKeyCaptureDialog — focus-scoped PTT key capture (documented limitation) - PerUserTuningDialog — real-time gain/mute/NR applied to all of a user's streams - Accessibility: explicit AccessibleName/Description on every control, & mnemonics, Activity log ListBox as durable screen-reader record, AutomationNotification for curated live announcements Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:35:16 +02:00
delete[] list->items;
list->items = nullptr;
list->count = 0;
}
vc_result vc_list_users(vc_client* c, vc_user_list* out) {
if (c == nullptr || out == nullptr) return VC_ERR_INVALID_ARG;
return c->list_users(out);
}
void vc_free_user_list(vc_user_list* list) {
if (list == nullptr || list->items == nullptr) return;
for (size_t i = 0; i < list->count; ++i) delete[] list->items[i].nickname;
delete[] list->items;
list->items = nullptr;
list->count = 0;
}
vc_result vc_list_user_streams(vc_client* c, uint32_t user_id, vc_stream_summary_list* out) {
if (c == nullptr || out == nullptr) return VC_ERR_INVALID_ARG;
return c->list_user_streams(user_id, out);
}
void vc_free_stream_summary_list(vc_stream_summary_list* list) {
if (list == nullptr || list->items == nullptr) return;
for (size_t i = 0; i < list->count; ++i) delete[] list->items[i].label;
delete[] list->items;
list->items = nullptr;
list->count = 0;
}
vc_result vc_confirm_server_identity(vc_client* c, int accept) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->confirm_server_identity(accept != 0);
}
vc_result vc_get_server_identity_display(vc_client* c, char* out_buf, size_t buf_cap,
size_t* out_len) {
feat(M4): Windows WinForms client, TOFU identity pinning, VAD threshold + always-on mode Core ABI extensions (voicecat.h): - vc_list_channels / vc_list_users / vc_list_user_streams — pull-based snapshot getters for the channel-tree and user-list UI; session_model_mu_ guards cross-thread reads - VC_EVENT_JOIN_RESULT / vc_join_channel — channel join with optional password - VC_EVENT_SERVER_IDENTITY + vc_confirm_server_identity — TOFU gate that blocks io_thread_ until the UI approves or rejects; pins TLS leaf-cert SHA-256 (not declared Ed25519) - vc_get_server_identity_display — Ed25519 fingerprint for human-readable display only - VC_INPUT_ALWAYS_ON = 2 in vc_input_mode — transmit unconditionally, no VAD gate - vc_set_vad_threshold — live RMS threshold update (0.0–1.0); EnergyVadProcessor stores it atomically so the audio RT path reads without a lock C++ implementation: - SessionModel::apply_snapshot / apply_channel_event fixed to populate parent_id, password_protected, and max_users (were permanently zeroed) - TlsContext::peer_cert_fingerprint — SHA-256 of peer leaf cert DER via mbedTLS - TofuStore split into peek (read-only) + pin (write) so first-connect only persists after user approval; tofu_store_path in vc_config for per-user pin file location - TcpAcceptor uses dual-stack IPv6+IPv4 fallback (fixes localhost → ::1 on Windows) - windows-client CMake preset: Release shared DLL, static MinGW runtime, no tools/tests - New C++ tests: test_channel_user_list_abi, test_tofu_flow (14/14 green) Windows client (clients/windows/ — .NET 10 WinForms): - VoiceCat.Interop: LibraryImport P/Invoke surface, UnmanagedCallersOnly callbacks, Channel<VoiceCatEvent> event delivery drained by 30ms WinForms Timer - VoiceCat.App: ConnectDialog (saved servers, DPAPI password storage), ServerIdentity- Dialog (TOFU first-connect / mismatch warning), MainForm (channel TreeView, user ListBox, RichTextBox chat, voice controls, device pickers, VAD/PTT/always-on mode, per-user gain/mute/NR tuning, VAD sensitivity TrackBar, level meter ProgressBar) - PttKeyCaptureDialog — focus-scoped PTT key capture (documented limitation) - PerUserTuningDialog — real-time gain/mute/NR applied to all of a user's streams - Accessibility: explicit AccessibleName/Description on every control, & mnemonics, Activity log ListBox as durable screen-reader record, AutomationNotification for curated live announcements Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:35:16 +02:00
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->get_server_identity_display(out_buf, buf_cap, out_len);
}
vc_result vc_kick_user(vc_client* c, uint32_t user_id, const char* reason) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->kick_user(user_id, reason);
}
vc_result vc_ban_user(vc_client* c, uint32_t user_id, const char* reason,
uint64_t expires_unix_ms) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->ban_user(user_id, reason, expires_unix_ms);
}
vc_result vc_set_permission(vc_client* c, uint32_t user_id, const vc_permissions* perms) {
if (c == nullptr || perms == nullptr) return VC_ERR_INVALID_ARG;
return c->set_permission(user_id, perms);
}
vc_result vc_set_server_mute(vc_client* c, uint32_t user_id, int muted, int deafened) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->set_server_mute(user_id, muted != 0, deafened != 0);
}
vc_result vc_move_user(vc_client* c, uint32_t user_id, uint32_t channel_id) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->move_user(user_id, channel_id);
}
vc_result vc_create_channel(vc_client* c, const vc_channel_info* info) {
if (c == nullptr || info == nullptr) return VC_ERR_INVALID_ARG;
return c->create_channel(info);
}
vc_result vc_edit_channel(vc_client* c, const vc_channel_info* info) {
if (c == nullptr || info == nullptr) return VC_ERR_INVALID_ARG;
return c->edit_channel(info);
}
vc_result vc_delete_channel(vc_client* c, uint32_t channel_id) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->delete_channel(channel_id);
}
vc_result vc_create_account(vc_client* c, const char* username, const char* password) {
if (c == nullptr || username == nullptr || password == nullptr) return VC_ERR_INVALID_ARG;
return c->create_account(username, password);
}
vc_result vc_reset_password(vc_client* c, const char* username, const char* new_password) {
if (c == nullptr || username == nullptr || new_password == nullptr) return VC_ERR_INVALID_ARG;
return c->reset_password(username, new_password);
}
vc_result vc_delete_account(vc_client* c, const char* username) {
if (c == nullptr || username == nullptr) return VC_ERR_INVALID_ARG;
return c->delete_account(username);
}
vc_result vc_list_accounts(vc_client* c) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->list_accounts();
}
vc_result vc_get_account_list(vc_client* c, vc_account_list* out) {
if (c == nullptr || out == nullptr) return VC_ERR_INVALID_ARG;
return c->get_account_list(out);
}
void vc_free_account_list(vc_account_list* list) {
if (list == nullptr || list->items == nullptr) return;
for (size_t i = 0; i < list->count; ++i) delete[] list->items[i].username;
delete[] list->items;
list->items = nullptr;
list->count = 0;
}
vc_result vc_get_permissions(vc_client* c, vc_permissions* out) {
if (c == nullptr || out == nullptr) return VC_ERR_INVALID_ARG;
return c->get_permissions(out);
}
vc_result vc_audio_suspend(vc_client* c) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->audio_suspend();
}
vc_result vc_audio_resume(vc_client* c) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->audio_resume();
}
fix(ios): stereo mic + A2DP output, add vc_audio_restart ABI Diagnosed by comparing against TeamTalk5 (Client/iTeamTalk), which achieves stereo mic + A2DP output. Five fixes: 1. configureStereoCapture now calls setPreferredInput + setInputDataSource (mirroring TeamTalk5's SoundDevicesModel). Previously omitted based on incorrect diagnosis that setPreferredInput collapsed A2DP — the real culprit was setPreferredInputNumberOfChannels(2), which neither project uses. 2. New C ABI: vc_audio_restart (full stop + re-init, unlike suspend/resume which only stop/start). Swift wrapper added. The withAudioSuspend wrapper that used it was removed after on-device testing showed it killed all audio (including VoiceOver) when switching presets — the core's set_capture_channels handles engine restart internally. 3. Bluetooth options: Voice Chat preset now includes BOTH .allowBluetoothHFP AND .allowBluetoothA2DP (matching TeamTalk5's UtilSound.swift:228). Previously HFP-only blocked A2DP headphones. 4. Capture channels now reset when switching stereo→mono via selectCaptureChannels/applyPreset. AudioSessionManager tracks activeMicStreamId (set by SessionState on join/leave voice). 5. Docs synced: voice.md, tech-stack.md, architecture.md, PROGRESS.md. Removed stale setPreferredInputNumberOfChannels(2) references. Verified: ctest --preset dev 21/21 green, iOS client builds. Stereo mic + A2DP output still needs on-device debugging — the core recipe is correct but iOS 26 route behavior requires hands-on testing with a debugger.
2026-06-19 16:58:21 +02:00
vc_result vc_audio_restart(vc_client* c) {
if (c == nullptr) return VC_ERR_INVALID_ARG;
return c->audio_restart();
}
} // extern "C"