feat(M3): multi-stream & per-channel tuning
Implements docs/roadmap.md M3: multiple concurrent streams per user (MIC +
SCREEN_AUDIO + AUX_DEVICE), independent per-stream receiver gain/mute/noise-
reduction, talk indicators, and enforced per-channel Opus configurability
(mono/stereo, bitrate, frame size, FEC/DTX, application).
Bugs fixed along the way (found while implementing, not pre-existing scope):
- Server hard-coded stream_id=1 for every announce, so a second stream from
the same user silently overwrote the first in SessionRegistry::set_user_stream.
Now a per-session counter (ConnSession::next_stream_id_); handle_stream_stop
validates against announced_stream_ids_ before clearing.
- Client dropped mode/dtx/complexity/application from effective_audio even for
the single M2 stream -- only sample_rate/bitrate_bps/frame_ms/fec were ever
applied to OpusParams. Fixed on both the send (handle_stream_announce_result)
and receive (sync_remote_streams) paths via a shared
opus_params_from_audio_config() helper.
- OpusEncoder always used OPUS_APPLICATION_VOIP; added OpusParams::application
and wired it through.
- on_playback's per-stream decode passed the wrong frame_size to opus_decode
(total samples instead of samples-per-channel), which would have overflowed
the decode buffer for any stereo stream.
- teardown_voice() raced when called concurrently from run_io()'s own cleanup
and from disconnect() on a different thread -- both could see
udp_thread_/talk_timer_thread_ as joinable() at once and race to join() the
same std::thread (intermittent std::system_error under ctest). Fixed with a
teardown_mu_ guard instead of carrying the flake forward.
New:
- Per-channel AudioConfig: SessionRegistry now seeds Lobby (mono/24kbps/VOIP/
FEC+DTX) and a new "Music Room" channel (stereo/128kbps/AUDIO/no DTX);
handle_stream_announce enforces the channel's config, clamping (not
overriding) bitrate_bps to its ceiling.
- core/src/core/client.h/.cpp: local-stream state is now a
std::unordered_map<int, LocalStream> keyed by vc_stream_kind, with
request_id-correlated announce/result handling (request_id already
round-tripped on the wire; just wasn't read before). on_capture_frame is
kind-aware and upmixes mono capture to stereo when a stream's config calls
for it. set_self_mute's mic_muted now only gates the MIC kind. NS is wired
through set_remote_stream. New run_talk_timer() thread emits
VC_EVENT_TALK_STATE from both remote and local edge detection.
- core/src/audio/audio_engine.h/.cpp: kind-keyed injection taps
(inject_capture), stereo-to-mono downmix at the decode/mix boundary,
RemoteStream gains recv_ns (lazy ApmProcessor) + noise_reduction_enabled
and last_voice_ms/talking; new set_stream_noise_reduction() and
poll_talk_transitions().
- core/src/session/session.h/.cpp: Stream now carries the full AudioConfig,
not just sample_rate/frame_ms.
- New additive C ABI (core/include/voicecat.h): vc_audio_config +
vc_get_stream_audio_config (effective Opus config for any stream you own or
a peer's); vc_test_inject_capture (test-only synthetic PCM injection,
clearly marked, mirrors AudioEngine::inject_capture).
- tests/test_m3_multistream.cpp: the M3 exit criterion through the real ABI
(mirrors test_voice_client_abi.cpp's approach, not raw sockets) -- two
concurrent local streams, independent gain/mute/NS control, per-channel
config divergence via vc_get_stream_audio_config, talk indicators.
Explicitly out of scope for this pass (tracked in PROGRESS.md, not silently
dropped): VAD/PTT input gate + device enumeration; real WASAPI loopback
capture for SCREEN_AUDIO (synthetic injection only); true stereo playback
output (AudioEngine's mixer/output device stays mono -- Opus itself is fully
stereo-correct on the wire).
ctest --test-dir build/m1-dev: 11/11 green, verified across 3 consecutive
full-suite runs plus 8 standalone runs of the new test.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 14:12:37 +02:00
|
|
|
/*
|
|
|
|
|
* test_m3_multistream — M3 exit criterion, exercised through the real C ABI.
|
|
|
|
|
*
|
|
|
|
|
* Mirrors test_voice_client_abi.cpp's approach (real vc_client instances, not raw sockets —
|
|
|
|
|
* the M2 lesson is that ABI-level coverage is what actually proves the client library works).
|
|
|
|
|
* Covers the whole M3 milestone in one flow:
|
|
|
|
|
*
|
|
|
|
|
* 1. A starts two concurrent local streams (MIC + SCREEN_AUDIO) -- distinct stream ids,
|
|
|
|
|
* both visible to B as separate STREAM_STARTED events for the same user.
|
|
|
|
|
* 2. Synthetic PCM (vc_test_inject_capture) flows into both of A's streams without crashing
|
|
|
|
|
* and without disrupting the control/voice plane; B observes a VC_EVENT_TALK_STATE
|
|
|
|
|
* talking=true edge for A's MIC stream while both are still in the same channel (voice
|
|
|
|
|
* only relays within a channel, so this must happen before step 4 moves A elsewhere).
|
|
|
|
|
* 3. B independently gains/mutes/NS-toggles A's two streams (vc_set_remote_stream) --
|
|
|
|
|
* one call doesn't clobber the other's routing; a bogus stream_id is rejected.
|
|
|
|
|
* 4. Per-channel Opus configurability: A joins "Music Room" (channel 2, stereo/128kbps/
|
|
|
|
|
* OPUS_AUDIO/no DTX) before announcing there, while B stays in "Lobby" (channel 1,
|
|
|
|
|
* mono/24kbps/OPUS_VOIP/DTX) -- vc_get_stream_audio_config shows the two streams'
|
|
|
|
|
* effective config differs exactly as the server enforces it.
|
|
|
|
|
*/
|
|
|
|
|
#include <cstdio>
|
|
|
|
|
|
|
|
|
|
#ifdef VOICECAT_HAS_NET
|
|
|
|
|
|
|
|
|
|
#include <atomic>
|
|
|
|
|
#include <chrono>
|
|
|
|
|
#include <cmath>
|
|
|
|
|
#include <condition_variable>
|
|
|
|
|
#include <filesystem>
|
|
|
|
|
#include <mutex>
|
|
|
|
|
#include <string>
|
|
|
|
|
#include <thread>
|
|
|
|
|
#include <vector>
|
|
|
|
|
|
|
|
|
|
#include "voicecat.h"
|
|
|
|
|
#include "server.h"
|
|
|
|
|
#include "db.h"
|
|
|
|
|
|
|
|
|
|
// ── Event tracking ────────────────────────────────────────────────────────────
|
|
|
|
|
|
|
|
|
|
struct StreamEvent {
|
|
|
|
|
bool started; // true = STARTED, false = STOPPED
|
|
|
|
|
uint32_t user_id;
|
|
|
|
|
uint32_t stream_id;
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
struct TalkEvent {
|
|
|
|
|
uint32_t user_id;
|
|
|
|
|
uint32_t stream_id;
|
|
|
|
|
bool talking;
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
struct EventStore {
|
|
|
|
|
std::mutex mu;
|
|
|
|
|
std::condition_variable cv;
|
|
|
|
|
|
|
|
|
|
bool auth_ok{false};
|
|
|
|
|
uint32_t self_user_id{0};
|
|
|
|
|
bool channel_list_received{false};
|
|
|
|
|
std::vector<StreamEvent> stream_events;
|
|
|
|
|
std::vector<TalkEvent> talk_events;
|
feat: fix voice join/leave, channel edit defaults, channel-update stream restart
Three bugs fixed across the full stack (proto/server/core/ABI/Win/macOS/iOS):
1. Join/Leave Voice now truly subscribes/unsubscribes from the voice plane.
Previously the button only toggled the local mic — receiving was always on
(gated by channel membership alone). Added a protocol-level voice subscription
concept: new SubscribeVoiceRequest/UnsubscribeVoiceRequest/VoiceSubscriptionResult
proto messages, User.voice_subscribed field, vc_join_voice/vc_leave_voice C ABI
functions, VC_EVENT_VOICE_STATE event, server-side voice_subscribed flag checked
by the SFU relay recipient filter, and core-client gating of remote-stream
decoder setup. All three clients rewired to subscribe+mic on Join / unsubscribe
on Leave. Text chat works regardless of voice subscription.
2. Channel edit dialog now shows the channel's actual current settings. The read
struct vc_channel was missing sort_order and audio fields — only the write
struct vc_channel_info had them. Extended vc_channel with both (additive, no
ABI break), updated the session model and list_channels marshaling to populate
them, and updated all three clients' edit callers to use actual channel info
instead of hardcoded defaults.
3. Channel parameter updates now automatically restart everyone's streams.
Previously editing a channel's audio config persisted and broadcast a
ChannelEvent::UPDATED, but no layer restarted streams — encoders/decoders are
frozen at announce time. handle_channel_event now detects audio-config changes
on the user's current channel and stop->starts each active local stream. The
server reads the updated config on re-announce; peers wire up fresh decoders
at the new ssrc.
All 29 CTest tests pass; Windows DLL + C# client build clean. Apple clients not
yet compile-verified (Windows environment).
2026-06-24 14:29:39 +02:00
|
|
|
bool voice_subscribed{false};
|
feat(M3): multi-stream & per-channel tuning
Implements docs/roadmap.md M3: multiple concurrent streams per user (MIC +
SCREEN_AUDIO + AUX_DEVICE), independent per-stream receiver gain/mute/noise-
reduction, talk indicators, and enforced per-channel Opus configurability
(mono/stereo, bitrate, frame size, FEC/DTX, application).
Bugs fixed along the way (found while implementing, not pre-existing scope):
- Server hard-coded stream_id=1 for every announce, so a second stream from
the same user silently overwrote the first in SessionRegistry::set_user_stream.
Now a per-session counter (ConnSession::next_stream_id_); handle_stream_stop
validates against announced_stream_ids_ before clearing.
- Client dropped mode/dtx/complexity/application from effective_audio even for
the single M2 stream -- only sample_rate/bitrate_bps/frame_ms/fec were ever
applied to OpusParams. Fixed on both the send (handle_stream_announce_result)
and receive (sync_remote_streams) paths via a shared
opus_params_from_audio_config() helper.
- OpusEncoder always used OPUS_APPLICATION_VOIP; added OpusParams::application
and wired it through.
- on_playback's per-stream decode passed the wrong frame_size to opus_decode
(total samples instead of samples-per-channel), which would have overflowed
the decode buffer for any stereo stream.
- teardown_voice() raced when called concurrently from run_io()'s own cleanup
and from disconnect() on a different thread -- both could see
udp_thread_/talk_timer_thread_ as joinable() at once and race to join() the
same std::thread (intermittent std::system_error under ctest). Fixed with a
teardown_mu_ guard instead of carrying the flake forward.
New:
- Per-channel AudioConfig: SessionRegistry now seeds Lobby (mono/24kbps/VOIP/
FEC+DTX) and a new "Music Room" channel (stereo/128kbps/AUDIO/no DTX);
handle_stream_announce enforces the channel's config, clamping (not
overriding) bitrate_bps to its ceiling.
- core/src/core/client.h/.cpp: local-stream state is now a
std::unordered_map<int, LocalStream> keyed by vc_stream_kind, with
request_id-correlated announce/result handling (request_id already
round-tripped on the wire; just wasn't read before). on_capture_frame is
kind-aware and upmixes mono capture to stereo when a stream's config calls
for it. set_self_mute's mic_muted now only gates the MIC kind. NS is wired
through set_remote_stream. New run_talk_timer() thread emits
VC_EVENT_TALK_STATE from both remote and local edge detection.
- core/src/audio/audio_engine.h/.cpp: kind-keyed injection taps
(inject_capture), stereo-to-mono downmix at the decode/mix boundary,
RemoteStream gains recv_ns (lazy ApmProcessor) + noise_reduction_enabled
and last_voice_ms/talking; new set_stream_noise_reduction() and
poll_talk_transitions().
- core/src/session/session.h/.cpp: Stream now carries the full AudioConfig,
not just sample_rate/frame_ms.
- New additive C ABI (core/include/voicecat.h): vc_audio_config +
vc_get_stream_audio_config (effective Opus config for any stream you own or
a peer's); vc_test_inject_capture (test-only synthetic PCM injection,
clearly marked, mirrors AudioEngine::inject_capture).
- tests/test_m3_multistream.cpp: the M3 exit criterion through the real ABI
(mirrors test_voice_client_abi.cpp's approach, not raw sockets) -- two
concurrent local streams, independent gain/mute/NS control, per-channel
config divergence via vc_get_stream_audio_config, talk indicators.
Explicitly out of scope for this pass (tracked in PROGRESS.md, not silently
dropped): VAD/PTT input gate + device enumeration; real WASAPI loopback
capture for SCREEN_AUDIO (synthetic injection only); true stereo playback
output (AudioEngine's mixer/output device stays mono -- Opus itself is fully
stereo-correct on the wire).
ctest --test-dir build/m1-dev: 11/11 green, verified across 3 consecutive
full-suite runs plus 8 standalone runs of the new test.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 14:12:37 +02:00
|
|
|
bool disconnected{false};
|
|
|
|
|
|
|
|
|
|
const char* label{nullptr};
|
feat(M4): Windows WinForms client, TOFU identity pinning, VAD threshold + always-on mode
Core ABI extensions (voicecat.h):
- vc_list_channels / vc_list_users / vc_list_user_streams — pull-based snapshot getters
for the channel-tree and user-list UI; session_model_mu_ guards cross-thread reads
- VC_EVENT_JOIN_RESULT / vc_join_channel — channel join with optional password
- VC_EVENT_SERVER_IDENTITY + vc_confirm_server_identity — TOFU gate that blocks io_thread_
until the UI approves or rejects; pins TLS leaf-cert SHA-256 (not declared Ed25519)
- vc_get_server_identity_display — Ed25519 fingerprint for human-readable display only
- VC_INPUT_ALWAYS_ON = 2 in vc_input_mode — transmit unconditionally, no VAD gate
- vc_set_vad_threshold — live RMS threshold update (0.0–1.0); EnergyVadProcessor stores
it atomically so the audio RT path reads without a lock
C++ implementation:
- SessionModel::apply_snapshot / apply_channel_event fixed to populate parent_id,
password_protected, and max_users (were permanently zeroed)
- TlsContext::peer_cert_fingerprint — SHA-256 of peer leaf cert DER via mbedTLS
- TofuStore split into peek (read-only) + pin (write) so first-connect only persists
after user approval; tofu_store_path in vc_config for per-user pin file location
- TcpAcceptor uses dual-stack IPv6+IPv4 fallback (fixes localhost → ::1 on Windows)
- windows-client CMake preset: Release shared DLL, static MinGW runtime, no tools/tests
- New C++ tests: test_channel_user_list_abi, test_tofu_flow (14/14 green)
Windows client (clients/windows/ — .NET 10 WinForms):
- VoiceCat.Interop: LibraryImport P/Invoke surface, UnmanagedCallersOnly callbacks,
Channel<VoiceCatEvent> event delivery drained by 30ms WinForms Timer
- VoiceCat.App: ConnectDialog (saved servers, DPAPI password storage), ServerIdentity-
Dialog (TOFU first-connect / mismatch warning), MainForm (channel TreeView, user
ListBox, RichTextBox chat, voice controls, device pickers, VAD/PTT/always-on mode,
per-user gain/mute/NR tuning, VAD sensitivity TrackBar, level meter ProgressBar)
- PttKeyCaptureDialog — focus-scoped PTT key capture (documented limitation)
- PerUserTuningDialog — real-time gain/mute/NR applied to all of a user's streams
- Accessibility: explicit AccessibleName/Description on every control, & mnemonics,
Activity log ListBox as durable screen-reader record, AutomationNotification for
curated live announcements
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:35:16 +02:00
|
|
|
|
|
|
|
|
// Set right after vc_client_create, before vc_connect — lets on_event auto-confirm the
|
|
|
|
|
// M4 TOFU gate (VC_EVENT_SERVER_IDENTITY below) for this headless test.
|
|
|
|
|
vc_client* client{nullptr};
|
feat(M3): multi-stream & per-channel tuning
Implements docs/roadmap.md M3: multiple concurrent streams per user (MIC +
SCREEN_AUDIO + AUX_DEVICE), independent per-stream receiver gain/mute/noise-
reduction, talk indicators, and enforced per-channel Opus configurability
(mono/stereo, bitrate, frame size, FEC/DTX, application).
Bugs fixed along the way (found while implementing, not pre-existing scope):
- Server hard-coded stream_id=1 for every announce, so a second stream from
the same user silently overwrote the first in SessionRegistry::set_user_stream.
Now a per-session counter (ConnSession::next_stream_id_); handle_stream_stop
validates against announced_stream_ids_ before clearing.
- Client dropped mode/dtx/complexity/application from effective_audio even for
the single M2 stream -- only sample_rate/bitrate_bps/frame_ms/fec were ever
applied to OpusParams. Fixed on both the send (handle_stream_announce_result)
and receive (sync_remote_streams) paths via a shared
opus_params_from_audio_config() helper.
- OpusEncoder always used OPUS_APPLICATION_VOIP; added OpusParams::application
and wired it through.
- on_playback's per-stream decode passed the wrong frame_size to opus_decode
(total samples instead of samples-per-channel), which would have overflowed
the decode buffer for any stereo stream.
- teardown_voice() raced when called concurrently from run_io()'s own cleanup
and from disconnect() on a different thread -- both could see
udp_thread_/talk_timer_thread_ as joinable() at once and race to join() the
same std::thread (intermittent std::system_error under ctest). Fixed with a
teardown_mu_ guard instead of carrying the flake forward.
New:
- Per-channel AudioConfig: SessionRegistry now seeds Lobby (mono/24kbps/VOIP/
FEC+DTX) and a new "Music Room" channel (stereo/128kbps/AUDIO/no DTX);
handle_stream_announce enforces the channel's config, clamping (not
overriding) bitrate_bps to its ceiling.
- core/src/core/client.h/.cpp: local-stream state is now a
std::unordered_map<int, LocalStream> keyed by vc_stream_kind, with
request_id-correlated announce/result handling (request_id already
round-tripped on the wire; just wasn't read before). on_capture_frame is
kind-aware and upmixes mono capture to stereo when a stream's config calls
for it. set_self_mute's mic_muted now only gates the MIC kind. NS is wired
through set_remote_stream. New run_talk_timer() thread emits
VC_EVENT_TALK_STATE from both remote and local edge detection.
- core/src/audio/audio_engine.h/.cpp: kind-keyed injection taps
(inject_capture), stereo-to-mono downmix at the decode/mix boundary,
RemoteStream gains recv_ns (lazy ApmProcessor) + noise_reduction_enabled
and last_voice_ms/talking; new set_stream_noise_reduction() and
poll_talk_transitions().
- core/src/session/session.h/.cpp: Stream now carries the full AudioConfig,
not just sample_rate/frame_ms.
- New additive C ABI (core/include/voicecat.h): vc_audio_config +
vc_get_stream_audio_config (effective Opus config for any stream you own or
a peer's); vc_test_inject_capture (test-only synthetic PCM injection,
clearly marked, mirrors AudioEngine::inject_capture).
- tests/test_m3_multistream.cpp: the M3 exit criterion through the real ABI
(mirrors test_voice_client_abi.cpp's approach, not raw sockets) -- two
concurrent local streams, independent gain/mute/NS control, per-channel
config divergence via vc_get_stream_audio_config, talk indicators.
Explicitly out of scope for this pass (tracked in PROGRESS.md, not silently
dropped): VAD/PTT input gate + device enumeration; real WASAPI loopback
capture for SCREEN_AUDIO (synthetic injection only); true stereo playback
output (AudioEngine's mixer/output device stays mono -- Opus itself is fully
stereo-correct on the wire).
ctest --test-dir build/m1-dev: 11/11 green, verified across 3 consecutive
full-suite runs plus 8 standalone runs of the new test.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 14:12:37 +02:00
|
|
|
};
|
|
|
|
|
|
|
|
|
|
static void on_event(void* user, const vc_event* ev) {
|
|
|
|
|
auto* s = static_cast<EventStore*>(user);
|
|
|
|
|
std::lock_guard lk(s->mu);
|
|
|
|
|
switch (ev->type) {
|
feat(M4): Windows WinForms client, TOFU identity pinning, VAD threshold + always-on mode
Core ABI extensions (voicecat.h):
- vc_list_channels / vc_list_users / vc_list_user_streams — pull-based snapshot getters
for the channel-tree and user-list UI; session_model_mu_ guards cross-thread reads
- VC_EVENT_JOIN_RESULT / vc_join_channel — channel join with optional password
- VC_EVENT_SERVER_IDENTITY + vc_confirm_server_identity — TOFU gate that blocks io_thread_
until the UI approves or rejects; pins TLS leaf-cert SHA-256 (not declared Ed25519)
- vc_get_server_identity_display — Ed25519 fingerprint for human-readable display only
- VC_INPUT_ALWAYS_ON = 2 in vc_input_mode — transmit unconditionally, no VAD gate
- vc_set_vad_threshold — live RMS threshold update (0.0–1.0); EnergyVadProcessor stores
it atomically so the audio RT path reads without a lock
C++ implementation:
- SessionModel::apply_snapshot / apply_channel_event fixed to populate parent_id,
password_protected, and max_users (were permanently zeroed)
- TlsContext::peer_cert_fingerprint — SHA-256 of peer leaf cert DER via mbedTLS
- TofuStore split into peek (read-only) + pin (write) so first-connect only persists
after user approval; tofu_store_path in vc_config for per-user pin file location
- TcpAcceptor uses dual-stack IPv6+IPv4 fallback (fixes localhost → ::1 on Windows)
- windows-client CMake preset: Release shared DLL, static MinGW runtime, no tools/tests
- New C++ tests: test_channel_user_list_abi, test_tofu_flow (14/14 green)
Windows client (clients/windows/ — .NET 10 WinForms):
- VoiceCat.Interop: LibraryImport P/Invoke surface, UnmanagedCallersOnly callbacks,
Channel<VoiceCatEvent> event delivery drained by 30ms WinForms Timer
- VoiceCat.App: ConnectDialog (saved servers, DPAPI password storage), ServerIdentity-
Dialog (TOFU first-connect / mismatch warning), MainForm (channel TreeView, user
ListBox, RichTextBox chat, voice controls, device pickers, VAD/PTT/always-on mode,
per-user gain/mute/NR tuning, VAD sensitivity TrackBar, level meter ProgressBar)
- PttKeyCaptureDialog — focus-scoped PTT key capture (documented limitation)
- PerUserTuningDialog — real-time gain/mute/NR applied to all of a user's streams
- Accessibility: explicit AccessibleName/Description on every control, & mnemonics,
Activity log ListBox as durable screen-reader record, AutomationNotification for
curated live announcements
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:35:16 +02:00
|
|
|
case VC_EVENT_SERVER_IDENTITY:
|
|
|
|
|
// No human to ask in a headless test — trust on first connect unconditionally.
|
|
|
|
|
vc_confirm_server_identity(s->client, 1);
|
|
|
|
|
break;
|
feat(M3): multi-stream & per-channel tuning
Implements docs/roadmap.md M3: multiple concurrent streams per user (MIC +
SCREEN_AUDIO + AUX_DEVICE), independent per-stream receiver gain/mute/noise-
reduction, talk indicators, and enforced per-channel Opus configurability
(mono/stereo, bitrate, frame size, FEC/DTX, application).
Bugs fixed along the way (found while implementing, not pre-existing scope):
- Server hard-coded stream_id=1 for every announce, so a second stream from
the same user silently overwrote the first in SessionRegistry::set_user_stream.
Now a per-session counter (ConnSession::next_stream_id_); handle_stream_stop
validates against announced_stream_ids_ before clearing.
- Client dropped mode/dtx/complexity/application from effective_audio even for
the single M2 stream -- only sample_rate/bitrate_bps/frame_ms/fec were ever
applied to OpusParams. Fixed on both the send (handle_stream_announce_result)
and receive (sync_remote_streams) paths via a shared
opus_params_from_audio_config() helper.
- OpusEncoder always used OPUS_APPLICATION_VOIP; added OpusParams::application
and wired it through.
- on_playback's per-stream decode passed the wrong frame_size to opus_decode
(total samples instead of samples-per-channel), which would have overflowed
the decode buffer for any stereo stream.
- teardown_voice() raced when called concurrently from run_io()'s own cleanup
and from disconnect() on a different thread -- both could see
udp_thread_/talk_timer_thread_ as joinable() at once and race to join() the
same std::thread (intermittent std::system_error under ctest). Fixed with a
teardown_mu_ guard instead of carrying the flake forward.
New:
- Per-channel AudioConfig: SessionRegistry now seeds Lobby (mono/24kbps/VOIP/
FEC+DTX) and a new "Music Room" channel (stereo/128kbps/AUDIO/no DTX);
handle_stream_announce enforces the channel's config, clamping (not
overriding) bitrate_bps to its ceiling.
- core/src/core/client.h/.cpp: local-stream state is now a
std::unordered_map<int, LocalStream> keyed by vc_stream_kind, with
request_id-correlated announce/result handling (request_id already
round-tripped on the wire; just wasn't read before). on_capture_frame is
kind-aware and upmixes mono capture to stereo when a stream's config calls
for it. set_self_mute's mic_muted now only gates the MIC kind. NS is wired
through set_remote_stream. New run_talk_timer() thread emits
VC_EVENT_TALK_STATE from both remote and local edge detection.
- core/src/audio/audio_engine.h/.cpp: kind-keyed injection taps
(inject_capture), stereo-to-mono downmix at the decode/mix boundary,
RemoteStream gains recv_ns (lazy ApmProcessor) + noise_reduction_enabled
and last_voice_ms/talking; new set_stream_noise_reduction() and
poll_talk_transitions().
- core/src/session/session.h/.cpp: Stream now carries the full AudioConfig,
not just sample_rate/frame_ms.
- New additive C ABI (core/include/voicecat.h): vc_audio_config +
vc_get_stream_audio_config (effective Opus config for any stream you own or
a peer's); vc_test_inject_capture (test-only synthetic PCM injection,
clearly marked, mirrors AudioEngine::inject_capture).
- tests/test_m3_multistream.cpp: the M3 exit criterion through the real ABI
(mirrors test_voice_client_abi.cpp's approach, not raw sockets) -- two
concurrent local streams, independent gain/mute/NS control, per-channel
config divergence via vc_get_stream_audio_config, talk indicators.
Explicitly out of scope for this pass (tracked in PROGRESS.md, not silently
dropped): VAD/PTT input gate + device enumeration; real WASAPI loopback
capture for SCREEN_AUDIO (synthetic injection only); true stereo playback
output (AudioEngine's mixer/output device stays mono -- Opus itself is fully
stereo-correct on the wire).
ctest --test-dir build/m1-dev: 11/11 green, verified across 3 consecutive
full-suite runs plus 8 standalone runs of the new test.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 14:12:37 +02:00
|
|
|
case VC_EVENT_AUTH_RESULT:
|
|
|
|
|
s->auth_ok = (ev->result == VC_OK);
|
|
|
|
|
s->self_user_id = ev->user_id;
|
|
|
|
|
if (!s->auth_ok) std::fprintf(stderr, "[%s] AUTH FAILED: %s\n",
|
|
|
|
|
s->label ? s->label : "?", ev->text ? ev->text : "(no msg)");
|
|
|
|
|
break;
|
|
|
|
|
case VC_EVENT_CHANNEL_LIST:
|
|
|
|
|
s->channel_list_received = true;
|
|
|
|
|
break;
|
feat: fix voice join/leave, channel edit defaults, channel-update stream restart
Three bugs fixed across the full stack (proto/server/core/ABI/Win/macOS/iOS):
1. Join/Leave Voice now truly subscribes/unsubscribes from the voice plane.
Previously the button only toggled the local mic — receiving was always on
(gated by channel membership alone). Added a protocol-level voice subscription
concept: new SubscribeVoiceRequest/UnsubscribeVoiceRequest/VoiceSubscriptionResult
proto messages, User.voice_subscribed field, vc_join_voice/vc_leave_voice C ABI
functions, VC_EVENT_VOICE_STATE event, server-side voice_subscribed flag checked
by the SFU relay recipient filter, and core-client gating of remote-stream
decoder setup. All three clients rewired to subscribe+mic on Join / unsubscribe
on Leave. Text chat works regardless of voice subscription.
2. Channel edit dialog now shows the channel's actual current settings. The read
struct vc_channel was missing sort_order and audio fields — only the write
struct vc_channel_info had them. Extended vc_channel with both (additive, no
ABI break), updated the session model and list_channels marshaling to populate
them, and updated all three clients' edit callers to use actual channel info
instead of hardcoded defaults.
3. Channel parameter updates now automatically restart everyone's streams.
Previously editing a channel's audio config persisted and broadcast a
ChannelEvent::UPDATED, but no layer restarted streams — encoders/decoders are
frozen at announce time. handle_channel_event now detects audio-config changes
on the user's current channel and stop->starts each active local stream. The
server reads the updated config on re-announce; peers wire up fresh decoders
at the new ssrc.
All 29 CTest tests pass; Windows DLL + C# client build clean. Apple clients not
yet compile-verified (Windows environment).
2026-06-24 14:29:39 +02:00
|
|
|
case VC_EVENT_VOICE_STATE:
|
|
|
|
|
s->voice_subscribed = (ev->u32a == 1);
|
|
|
|
|
break;
|
feat(M3): multi-stream & per-channel tuning
Implements docs/roadmap.md M3: multiple concurrent streams per user (MIC +
SCREEN_AUDIO + AUX_DEVICE), independent per-stream receiver gain/mute/noise-
reduction, talk indicators, and enforced per-channel Opus configurability
(mono/stereo, bitrate, frame size, FEC/DTX, application).
Bugs fixed along the way (found while implementing, not pre-existing scope):
- Server hard-coded stream_id=1 for every announce, so a second stream from
the same user silently overwrote the first in SessionRegistry::set_user_stream.
Now a per-session counter (ConnSession::next_stream_id_); handle_stream_stop
validates against announced_stream_ids_ before clearing.
- Client dropped mode/dtx/complexity/application from effective_audio even for
the single M2 stream -- only sample_rate/bitrate_bps/frame_ms/fec were ever
applied to OpusParams. Fixed on both the send (handle_stream_announce_result)
and receive (sync_remote_streams) paths via a shared
opus_params_from_audio_config() helper.
- OpusEncoder always used OPUS_APPLICATION_VOIP; added OpusParams::application
and wired it through.
- on_playback's per-stream decode passed the wrong frame_size to opus_decode
(total samples instead of samples-per-channel), which would have overflowed
the decode buffer for any stereo stream.
- teardown_voice() raced when called concurrently from run_io()'s own cleanup
and from disconnect() on a different thread -- both could see
udp_thread_/talk_timer_thread_ as joinable() at once and race to join() the
same std::thread (intermittent std::system_error under ctest). Fixed with a
teardown_mu_ guard instead of carrying the flake forward.
New:
- Per-channel AudioConfig: SessionRegistry now seeds Lobby (mono/24kbps/VOIP/
FEC+DTX) and a new "Music Room" channel (stereo/128kbps/AUDIO/no DTX);
handle_stream_announce enforces the channel's config, clamping (not
overriding) bitrate_bps to its ceiling.
- core/src/core/client.h/.cpp: local-stream state is now a
std::unordered_map<int, LocalStream> keyed by vc_stream_kind, with
request_id-correlated announce/result handling (request_id already
round-tripped on the wire; just wasn't read before). on_capture_frame is
kind-aware and upmixes mono capture to stereo when a stream's config calls
for it. set_self_mute's mic_muted now only gates the MIC kind. NS is wired
through set_remote_stream. New run_talk_timer() thread emits
VC_EVENT_TALK_STATE from both remote and local edge detection.
- core/src/audio/audio_engine.h/.cpp: kind-keyed injection taps
(inject_capture), stereo-to-mono downmix at the decode/mix boundary,
RemoteStream gains recv_ns (lazy ApmProcessor) + noise_reduction_enabled
and last_voice_ms/talking; new set_stream_noise_reduction() and
poll_talk_transitions().
- core/src/session/session.h/.cpp: Stream now carries the full AudioConfig,
not just sample_rate/frame_ms.
- New additive C ABI (core/include/voicecat.h): vc_audio_config +
vc_get_stream_audio_config (effective Opus config for any stream you own or
a peer's); vc_test_inject_capture (test-only synthetic PCM injection,
clearly marked, mirrors AudioEngine::inject_capture).
- tests/test_m3_multistream.cpp: the M3 exit criterion through the real ABI
(mirrors test_voice_client_abi.cpp's approach, not raw sockets) -- two
concurrent local streams, independent gain/mute/NS control, per-channel
config divergence via vc_get_stream_audio_config, talk indicators.
Explicitly out of scope for this pass (tracked in PROGRESS.md, not silently
dropped): VAD/PTT input gate + device enumeration; real WASAPI loopback
capture for SCREEN_AUDIO (synthetic injection only); true stereo playback
output (AudioEngine's mixer/output device stays mono -- Opus itself is fully
stereo-correct on the wire).
ctest --test-dir build/m1-dev: 11/11 green, verified across 3 consecutive
full-suite runs plus 8 standalone runs of the new test.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 14:12:37 +02:00
|
|
|
case VC_EVENT_STREAM_STARTED:
|
|
|
|
|
s->stream_events.push_back({true, ev->user_id, ev->stream_id});
|
|
|
|
|
break;
|
|
|
|
|
case VC_EVENT_STREAM_STOPPED:
|
|
|
|
|
s->stream_events.push_back({false, ev->user_id, ev->stream_id});
|
|
|
|
|
break;
|
|
|
|
|
case VC_EVENT_TALK_STATE:
|
|
|
|
|
s->talk_events.push_back({ev->user_id, ev->stream_id, ev->u32a != 0});
|
|
|
|
|
break;
|
|
|
|
|
case VC_EVENT_ERROR:
|
|
|
|
|
std::fprintf(stderr, "[%s] ERROR rc=%d: %s\n",
|
|
|
|
|
s->label ? s->label : "?", ev->result, ev->text ? ev->text : "");
|
|
|
|
|
break;
|
|
|
|
|
case VC_EVENT_DISCONNECTED:
|
|
|
|
|
s->disconnected = true;
|
|
|
|
|
break;
|
|
|
|
|
default:
|
|
|
|
|
break;
|
|
|
|
|
}
|
|
|
|
|
s->cv.notify_all();
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
template <typename Pred>
|
|
|
|
|
static bool wait_for(EventStore& s, Pred pred, int timeout_ms) {
|
|
|
|
|
auto deadline = std::chrono::steady_clock::now() + std::chrono::milliseconds(timeout_ms);
|
|
|
|
|
std::unique_lock lk(s.mu);
|
|
|
|
|
return s.cv.wait_until(lk, deadline, [&] { return pred(s); });
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
static std::vector<int16_t> make_sine_frame(int frame_idx, float freq_hz,
|
|
|
|
|
int frame_samples = 960) {
|
|
|
|
|
std::vector<int16_t> pcm(frame_samples);
|
|
|
|
|
for (int i = 0; i < frame_samples; ++i) {
|
|
|
|
|
float t = static_cast<float>(frame_idx * frame_samples + i) / 48000.0f;
|
|
|
|
|
pcm[i] = static_cast<int16_t>(std::sin(2.0f * 3.14159265f * freq_hz * t) * 16000.0f);
|
|
|
|
|
}
|
|
|
|
|
return pcm;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// ── Test harness ──────────────────────────────────────────────────────────────
|
|
|
|
|
|
|
|
|
|
static int g_failures = 0;
|
|
|
|
|
#define CHECK(cond) \
|
|
|
|
|
do { \
|
|
|
|
|
if (!(cond)) { \
|
|
|
|
|
std::printf("FAIL: %s (%s:%d)\n", #cond, __FILE__, __LINE__); \
|
|
|
|
|
++g_failures; \
|
|
|
|
|
} \
|
|
|
|
|
} while (0)
|
|
|
|
|
|
|
|
|
|
int main() {
|
|
|
|
|
auto tmp = std::filesystem::temp_directory_path() /
|
|
|
|
|
("vctest_m3_" + std::to_string(
|
|
|
|
|
std::chrono::steady_clock::now().time_since_epoch().count()));
|
|
|
|
|
std::filesystem::create_directories(tmp);
|
|
|
|
|
std::string data_dir = tmp.string();
|
|
|
|
|
|
|
|
|
|
std::atomic<uint16_t> bound_port{0};
|
|
|
|
|
std::mutex ready_mu;
|
|
|
|
|
std::condition_variable ready_cv;
|
|
|
|
|
bool ready{false};
|
|
|
|
|
|
|
|
|
|
voicecat::server::Config cfg;
|
|
|
|
|
cfg.data_dir = data_dir;
|
|
|
|
|
cfg.bind_port = 0;
|
|
|
|
|
cfg.media_port = 0;
|
|
|
|
|
cfg.server_name = "VoiceCat-M3Test";
|
|
|
|
|
cfg.allow_guests = true;
|
|
|
|
|
cfg.on_ready = [&](uint16_t p) {
|
|
|
|
|
bound_port.store(p);
|
|
|
|
|
{ std::lock_guard lk(ready_mu); ready = true; }
|
|
|
|
|
ready_cv.notify_all();
|
|
|
|
|
};
|
|
|
|
|
|
|
|
|
|
voicecat::server::Server server(cfg);
|
|
|
|
|
std::thread server_thread([&] { server.run(); });
|
|
|
|
|
|
|
|
|
|
{
|
|
|
|
|
std::unique_lock lk(ready_mu);
|
|
|
|
|
bool ok = ready_cv.wait_for(lk, std::chrono::seconds(10), [&] { return ready; });
|
|
|
|
|
if (!ok) {
|
|
|
|
|
std::printf("FAIL: server did not become ready within 10s\n");
|
|
|
|
|
server.stop();
|
|
|
|
|
server_thread.join();
|
|
|
|
|
std::filesystem::remove_all(tmp);
|
|
|
|
|
return 1;
|
|
|
|
|
}
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
uint16_t port = bound_port.load();
|
|
|
|
|
std::printf("m3_multistream: server ready on :%u\n", port);
|
|
|
|
|
|
|
|
|
|
// ── Client A: guest "M3-A" ────────────────────────────────────────────────
|
|
|
|
|
EventStore evA;
|
|
|
|
|
evA.label = "clientA";
|
|
|
|
|
vc_callbacks cbA{on_event, nullptr, &evA};
|
|
|
|
|
vc_config cfgA{"test-clientA", "0.1", VC_LOG_OFF};
|
|
|
|
|
vc_client* clientA = vc_client_create(&cfgA, cbA);
|
|
|
|
|
CHECK(clientA != nullptr);
|
feat(M4): Windows WinForms client, TOFU identity pinning, VAD threshold + always-on mode
Core ABI extensions (voicecat.h):
- vc_list_channels / vc_list_users / vc_list_user_streams — pull-based snapshot getters
for the channel-tree and user-list UI; session_model_mu_ guards cross-thread reads
- VC_EVENT_JOIN_RESULT / vc_join_channel — channel join with optional password
- VC_EVENT_SERVER_IDENTITY + vc_confirm_server_identity — TOFU gate that blocks io_thread_
until the UI approves or rejects; pins TLS leaf-cert SHA-256 (not declared Ed25519)
- vc_get_server_identity_display — Ed25519 fingerprint for human-readable display only
- VC_INPUT_ALWAYS_ON = 2 in vc_input_mode — transmit unconditionally, no VAD gate
- vc_set_vad_threshold — live RMS threshold update (0.0–1.0); EnergyVadProcessor stores
it atomically so the audio RT path reads without a lock
C++ implementation:
- SessionModel::apply_snapshot / apply_channel_event fixed to populate parent_id,
password_protected, and max_users (were permanently zeroed)
- TlsContext::peer_cert_fingerprint — SHA-256 of peer leaf cert DER via mbedTLS
- TofuStore split into peek (read-only) + pin (write) so first-connect only persists
after user approval; tofu_store_path in vc_config for per-user pin file location
- TcpAcceptor uses dual-stack IPv6+IPv4 fallback (fixes localhost → ::1 on Windows)
- windows-client CMake preset: Release shared DLL, static MinGW runtime, no tools/tests
- New C++ tests: test_channel_user_list_abi, test_tofu_flow (14/14 green)
Windows client (clients/windows/ — .NET 10 WinForms):
- VoiceCat.Interop: LibraryImport P/Invoke surface, UnmanagedCallersOnly callbacks,
Channel<VoiceCatEvent> event delivery drained by 30ms WinForms Timer
- VoiceCat.App: ConnectDialog (saved servers, DPAPI password storage), ServerIdentity-
Dialog (TOFU first-connect / mismatch warning), MainForm (channel TreeView, user
ListBox, RichTextBox chat, voice controls, device pickers, VAD/PTT/always-on mode,
per-user gain/mute/NR tuning, VAD sensitivity TrackBar, level meter ProgressBar)
- PttKeyCaptureDialog — focus-scoped PTT key capture (documented limitation)
- PerUserTuningDialog — real-time gain/mute/NR applied to all of a user's streams
- Accessibility: explicit AccessibleName/Description on every control, & mnemonics,
Activity log ListBox as durable screen-reader record, AutomationNotification for
curated live announcements
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:35:16 +02:00
|
|
|
evA.client = clientA;
|
feat(M3): multi-stream & per-channel tuning
Implements docs/roadmap.md M3: multiple concurrent streams per user (MIC +
SCREEN_AUDIO + AUX_DEVICE), independent per-stream receiver gain/mute/noise-
reduction, talk indicators, and enforced per-channel Opus configurability
(mono/stereo, bitrate, frame size, FEC/DTX, application).
Bugs fixed along the way (found while implementing, not pre-existing scope):
- Server hard-coded stream_id=1 for every announce, so a second stream from
the same user silently overwrote the first in SessionRegistry::set_user_stream.
Now a per-session counter (ConnSession::next_stream_id_); handle_stream_stop
validates against announced_stream_ids_ before clearing.
- Client dropped mode/dtx/complexity/application from effective_audio even for
the single M2 stream -- only sample_rate/bitrate_bps/frame_ms/fec were ever
applied to OpusParams. Fixed on both the send (handle_stream_announce_result)
and receive (sync_remote_streams) paths via a shared
opus_params_from_audio_config() helper.
- OpusEncoder always used OPUS_APPLICATION_VOIP; added OpusParams::application
and wired it through.
- on_playback's per-stream decode passed the wrong frame_size to opus_decode
(total samples instead of samples-per-channel), which would have overflowed
the decode buffer for any stereo stream.
- teardown_voice() raced when called concurrently from run_io()'s own cleanup
and from disconnect() on a different thread -- both could see
udp_thread_/talk_timer_thread_ as joinable() at once and race to join() the
same std::thread (intermittent std::system_error under ctest). Fixed with a
teardown_mu_ guard instead of carrying the flake forward.
New:
- Per-channel AudioConfig: SessionRegistry now seeds Lobby (mono/24kbps/VOIP/
FEC+DTX) and a new "Music Room" channel (stereo/128kbps/AUDIO/no DTX);
handle_stream_announce enforces the channel's config, clamping (not
overriding) bitrate_bps to its ceiling.
- core/src/core/client.h/.cpp: local-stream state is now a
std::unordered_map<int, LocalStream> keyed by vc_stream_kind, with
request_id-correlated announce/result handling (request_id already
round-tripped on the wire; just wasn't read before). on_capture_frame is
kind-aware and upmixes mono capture to stereo when a stream's config calls
for it. set_self_mute's mic_muted now only gates the MIC kind. NS is wired
through set_remote_stream. New run_talk_timer() thread emits
VC_EVENT_TALK_STATE from both remote and local edge detection.
- core/src/audio/audio_engine.h/.cpp: kind-keyed injection taps
(inject_capture), stereo-to-mono downmix at the decode/mix boundary,
RemoteStream gains recv_ns (lazy ApmProcessor) + noise_reduction_enabled
and last_voice_ms/talking; new set_stream_noise_reduction() and
poll_talk_transitions().
- core/src/session/session.h/.cpp: Stream now carries the full AudioConfig,
not just sample_rate/frame_ms.
- New additive C ABI (core/include/voicecat.h): vc_audio_config +
vc_get_stream_audio_config (effective Opus config for any stream you own or
a peer's); vc_test_inject_capture (test-only synthetic PCM injection,
clearly marked, mirrors AudioEngine::inject_capture).
- tests/test_m3_multistream.cpp: the M3 exit criterion through the real ABI
(mirrors test_voice_client_abi.cpp's approach, not raw sockets) -- two
concurrent local streams, independent gain/mute/NS control, per-channel
config divergence via vc_get_stream_audio_config, talk indicators.
Explicitly out of scope for this pass (tracked in PROGRESS.md, not silently
dropped): VAD/PTT input gate + device enumeration; real WASAPI loopback
capture for SCREEN_AUDIO (synthetic injection only); true stereo playback
output (AudioEngine's mixer/output device stays mono -- Opus itself is fully
stereo-correct on the wire).
ctest --test-dir build/m1-dev: 11/11 green, verified across 3 consecutive
full-suite runs plus 8 standalone runs of the new test.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 14:12:37 +02:00
|
|
|
|
|
|
|
|
CHECK(vc_connect(clientA, "127.0.0.1", port) == VC_OK);
|
|
|
|
|
CHECK(vc_authenticate_guest(clientA, "M3-A") == VC_OK);
|
|
|
|
|
CHECK(wait_for(evA, [](EventStore& s) { return s.auth_ok; }, 8000));
|
|
|
|
|
CHECK(wait_for(evA, [](EventStore& s) { return s.channel_list_received; }, 3000));
|
|
|
|
|
|
feat: fix voice join/leave, channel edit defaults, channel-update stream restart
Three bugs fixed across the full stack (proto/server/core/ABI/Win/macOS/iOS):
1. Join/Leave Voice now truly subscribes/unsubscribes from the voice plane.
Previously the button only toggled the local mic — receiving was always on
(gated by channel membership alone). Added a protocol-level voice subscription
concept: new SubscribeVoiceRequest/UnsubscribeVoiceRequest/VoiceSubscriptionResult
proto messages, User.voice_subscribed field, vc_join_voice/vc_leave_voice C ABI
functions, VC_EVENT_VOICE_STATE event, server-side voice_subscribed flag checked
by the SFU relay recipient filter, and core-client gating of remote-stream
decoder setup. All three clients rewired to subscribe+mic on Join / unsubscribe
on Leave. Text chat works regardless of voice subscription.
2. Channel edit dialog now shows the channel's actual current settings. The read
struct vc_channel was missing sort_order and audio fields — only the write
struct vc_channel_info had them. Extended vc_channel with both (additive, no
ABI break), updated the session model and list_channels marshaling to populate
them, and updated all three clients' edit callers to use actual channel info
instead of hardcoded defaults.
3. Channel parameter updates now automatically restart everyone's streams.
Previously editing a channel's audio config persisted and broadcast a
ChannelEvent::UPDATED, but no layer restarted streams — encoders/decoders are
frozen at announce time. handle_channel_event now detects audio-config changes
on the user's current channel and stop->starts each active local stream. The
server reads the updated config on re-announce; peers wire up fresh decoders
at the new ssrc.
All 29 CTest tests pass; Windows DLL + C# client build clean. Apple clients not
yet compile-verified (Windows environment).
2026-06-24 14:29:39 +02:00
|
|
|
CHECK(vc_join_voice(clientA) == VC_OK);
|
|
|
|
|
CHECK(wait_for(evA, [](EventStore& s) { return s.voice_subscribed; }, 5000));
|
|
|
|
|
|
feat(M3): multi-stream & per-channel tuning
Implements docs/roadmap.md M3: multiple concurrent streams per user (MIC +
SCREEN_AUDIO + AUX_DEVICE), independent per-stream receiver gain/mute/noise-
reduction, talk indicators, and enforced per-channel Opus configurability
(mono/stereo, bitrate, frame size, FEC/DTX, application).
Bugs fixed along the way (found while implementing, not pre-existing scope):
- Server hard-coded stream_id=1 for every announce, so a second stream from
the same user silently overwrote the first in SessionRegistry::set_user_stream.
Now a per-session counter (ConnSession::next_stream_id_); handle_stream_stop
validates against announced_stream_ids_ before clearing.
- Client dropped mode/dtx/complexity/application from effective_audio even for
the single M2 stream -- only sample_rate/bitrate_bps/frame_ms/fec were ever
applied to OpusParams. Fixed on both the send (handle_stream_announce_result)
and receive (sync_remote_streams) paths via a shared
opus_params_from_audio_config() helper.
- OpusEncoder always used OPUS_APPLICATION_VOIP; added OpusParams::application
and wired it through.
- on_playback's per-stream decode passed the wrong frame_size to opus_decode
(total samples instead of samples-per-channel), which would have overflowed
the decode buffer for any stereo stream.
- teardown_voice() raced when called concurrently from run_io()'s own cleanup
and from disconnect() on a different thread -- both could see
udp_thread_/talk_timer_thread_ as joinable() at once and race to join() the
same std::thread (intermittent std::system_error under ctest). Fixed with a
teardown_mu_ guard instead of carrying the flake forward.
New:
- Per-channel AudioConfig: SessionRegistry now seeds Lobby (mono/24kbps/VOIP/
FEC+DTX) and a new "Music Room" channel (stereo/128kbps/AUDIO/no DTX);
handle_stream_announce enforces the channel's config, clamping (not
overriding) bitrate_bps to its ceiling.
- core/src/core/client.h/.cpp: local-stream state is now a
std::unordered_map<int, LocalStream> keyed by vc_stream_kind, with
request_id-correlated announce/result handling (request_id already
round-tripped on the wire; just wasn't read before). on_capture_frame is
kind-aware and upmixes mono capture to stereo when a stream's config calls
for it. set_self_mute's mic_muted now only gates the MIC kind. NS is wired
through set_remote_stream. New run_talk_timer() thread emits
VC_EVENT_TALK_STATE from both remote and local edge detection.
- core/src/audio/audio_engine.h/.cpp: kind-keyed injection taps
(inject_capture), stereo-to-mono downmix at the decode/mix boundary,
RemoteStream gains recv_ns (lazy ApmProcessor) + noise_reduction_enabled
and last_voice_ms/talking; new set_stream_noise_reduction() and
poll_talk_transitions().
- core/src/session/session.h/.cpp: Stream now carries the full AudioConfig,
not just sample_rate/frame_ms.
- New additive C ABI (core/include/voicecat.h): vc_audio_config +
vc_get_stream_audio_config (effective Opus config for any stream you own or
a peer's); vc_test_inject_capture (test-only synthetic PCM injection,
clearly marked, mirrors AudioEngine::inject_capture).
- tests/test_m3_multistream.cpp: the M3 exit criterion through the real ABI
(mirrors test_voice_client_abi.cpp's approach, not raw sockets) -- two
concurrent local streams, independent gain/mute/NS control, per-channel
config divergence via vc_get_stream_audio_config, talk indicators.
Explicitly out of scope for this pass (tracked in PROGRESS.md, not silently
dropped): VAD/PTT input gate + device enumeration; real WASAPI loopback
capture for SCREEN_AUDIO (synthetic injection only); true stereo playback
output (AudioEngine's mixer/output device stays mono -- Opus itself is fully
stereo-correct on the wire).
ctest --test-dir build/m1-dev: 11/11 green, verified across 3 consecutive
full-suite runs plus 8 standalone runs of the new test.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 14:12:37 +02:00
|
|
|
// ── Client B: guest "M3-B" ────────────────────────────────────────────────
|
|
|
|
|
EventStore evB;
|
|
|
|
|
evB.label = "clientB";
|
|
|
|
|
vc_callbacks cbB{on_event, nullptr, &evB};
|
|
|
|
|
vc_config cfgB{"test-clientB", "0.1", VC_LOG_OFF};
|
|
|
|
|
vc_client* clientB = vc_client_create(&cfgB, cbB);
|
|
|
|
|
CHECK(clientB != nullptr);
|
feat(M4): Windows WinForms client, TOFU identity pinning, VAD threshold + always-on mode
Core ABI extensions (voicecat.h):
- vc_list_channels / vc_list_users / vc_list_user_streams — pull-based snapshot getters
for the channel-tree and user-list UI; session_model_mu_ guards cross-thread reads
- VC_EVENT_JOIN_RESULT / vc_join_channel — channel join with optional password
- VC_EVENT_SERVER_IDENTITY + vc_confirm_server_identity — TOFU gate that blocks io_thread_
until the UI approves or rejects; pins TLS leaf-cert SHA-256 (not declared Ed25519)
- vc_get_server_identity_display — Ed25519 fingerprint for human-readable display only
- VC_INPUT_ALWAYS_ON = 2 in vc_input_mode — transmit unconditionally, no VAD gate
- vc_set_vad_threshold — live RMS threshold update (0.0–1.0); EnergyVadProcessor stores
it atomically so the audio RT path reads without a lock
C++ implementation:
- SessionModel::apply_snapshot / apply_channel_event fixed to populate parent_id,
password_protected, and max_users (were permanently zeroed)
- TlsContext::peer_cert_fingerprint — SHA-256 of peer leaf cert DER via mbedTLS
- TofuStore split into peek (read-only) + pin (write) so first-connect only persists
after user approval; tofu_store_path in vc_config for per-user pin file location
- TcpAcceptor uses dual-stack IPv6+IPv4 fallback (fixes localhost → ::1 on Windows)
- windows-client CMake preset: Release shared DLL, static MinGW runtime, no tools/tests
- New C++ tests: test_channel_user_list_abi, test_tofu_flow (14/14 green)
Windows client (clients/windows/ — .NET 10 WinForms):
- VoiceCat.Interop: LibraryImport P/Invoke surface, UnmanagedCallersOnly callbacks,
Channel<VoiceCatEvent> event delivery drained by 30ms WinForms Timer
- VoiceCat.App: ConnectDialog (saved servers, DPAPI password storage), ServerIdentity-
Dialog (TOFU first-connect / mismatch warning), MainForm (channel TreeView, user
ListBox, RichTextBox chat, voice controls, device pickers, VAD/PTT/always-on mode,
per-user gain/mute/NR tuning, VAD sensitivity TrackBar, level meter ProgressBar)
- PttKeyCaptureDialog — focus-scoped PTT key capture (documented limitation)
- PerUserTuningDialog — real-time gain/mute/NR applied to all of a user's streams
- Accessibility: explicit AccessibleName/Description on every control, & mnemonics,
Activity log ListBox as durable screen-reader record, AutomationNotification for
curated live announcements
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:35:16 +02:00
|
|
|
evB.client = clientB;
|
feat(M3): multi-stream & per-channel tuning
Implements docs/roadmap.md M3: multiple concurrent streams per user (MIC +
SCREEN_AUDIO + AUX_DEVICE), independent per-stream receiver gain/mute/noise-
reduction, talk indicators, and enforced per-channel Opus configurability
(mono/stereo, bitrate, frame size, FEC/DTX, application).
Bugs fixed along the way (found while implementing, not pre-existing scope):
- Server hard-coded stream_id=1 for every announce, so a second stream from
the same user silently overwrote the first in SessionRegistry::set_user_stream.
Now a per-session counter (ConnSession::next_stream_id_); handle_stream_stop
validates against announced_stream_ids_ before clearing.
- Client dropped mode/dtx/complexity/application from effective_audio even for
the single M2 stream -- only sample_rate/bitrate_bps/frame_ms/fec were ever
applied to OpusParams. Fixed on both the send (handle_stream_announce_result)
and receive (sync_remote_streams) paths via a shared
opus_params_from_audio_config() helper.
- OpusEncoder always used OPUS_APPLICATION_VOIP; added OpusParams::application
and wired it through.
- on_playback's per-stream decode passed the wrong frame_size to opus_decode
(total samples instead of samples-per-channel), which would have overflowed
the decode buffer for any stereo stream.
- teardown_voice() raced when called concurrently from run_io()'s own cleanup
and from disconnect() on a different thread -- both could see
udp_thread_/talk_timer_thread_ as joinable() at once and race to join() the
same std::thread (intermittent std::system_error under ctest). Fixed with a
teardown_mu_ guard instead of carrying the flake forward.
New:
- Per-channel AudioConfig: SessionRegistry now seeds Lobby (mono/24kbps/VOIP/
FEC+DTX) and a new "Music Room" channel (stereo/128kbps/AUDIO/no DTX);
handle_stream_announce enforces the channel's config, clamping (not
overriding) bitrate_bps to its ceiling.
- core/src/core/client.h/.cpp: local-stream state is now a
std::unordered_map<int, LocalStream> keyed by vc_stream_kind, with
request_id-correlated announce/result handling (request_id already
round-tripped on the wire; just wasn't read before). on_capture_frame is
kind-aware and upmixes mono capture to stereo when a stream's config calls
for it. set_self_mute's mic_muted now only gates the MIC kind. NS is wired
through set_remote_stream. New run_talk_timer() thread emits
VC_EVENT_TALK_STATE from both remote and local edge detection.
- core/src/audio/audio_engine.h/.cpp: kind-keyed injection taps
(inject_capture), stereo-to-mono downmix at the decode/mix boundary,
RemoteStream gains recv_ns (lazy ApmProcessor) + noise_reduction_enabled
and last_voice_ms/talking; new set_stream_noise_reduction() and
poll_talk_transitions().
- core/src/session/session.h/.cpp: Stream now carries the full AudioConfig,
not just sample_rate/frame_ms.
- New additive C ABI (core/include/voicecat.h): vc_audio_config +
vc_get_stream_audio_config (effective Opus config for any stream you own or
a peer's); vc_test_inject_capture (test-only synthetic PCM injection,
clearly marked, mirrors AudioEngine::inject_capture).
- tests/test_m3_multistream.cpp: the M3 exit criterion through the real ABI
(mirrors test_voice_client_abi.cpp's approach, not raw sockets) -- two
concurrent local streams, independent gain/mute/NS control, per-channel
config divergence via vc_get_stream_audio_config, talk indicators.
Explicitly out of scope for this pass (tracked in PROGRESS.md, not silently
dropped): VAD/PTT input gate + device enumeration; real WASAPI loopback
capture for SCREEN_AUDIO (synthetic injection only); true stereo playback
output (AudioEngine's mixer/output device stays mono -- Opus itself is fully
stereo-correct on the wire).
ctest --test-dir build/m1-dev: 11/11 green, verified across 3 consecutive
full-suite runs plus 8 standalone runs of the new test.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 14:12:37 +02:00
|
|
|
|
|
|
|
|
CHECK(vc_connect(clientB, "127.0.0.1", port) == VC_OK);
|
|
|
|
|
CHECK(vc_authenticate_guest(clientB, "M3-B") == VC_OK);
|
|
|
|
|
CHECK(wait_for(evB, [](EventStore& s) { return s.auth_ok; }, 8000));
|
|
|
|
|
CHECK(wait_for(evB, [](EventStore& s) { return s.channel_list_received; }, 3000));
|
|
|
|
|
|
feat: fix voice join/leave, channel edit defaults, channel-update stream restart
Three bugs fixed across the full stack (proto/server/core/ABI/Win/macOS/iOS):
1. Join/Leave Voice now truly subscribes/unsubscribes from the voice plane.
Previously the button only toggled the local mic — receiving was always on
(gated by channel membership alone). Added a protocol-level voice subscription
concept: new SubscribeVoiceRequest/UnsubscribeVoiceRequest/VoiceSubscriptionResult
proto messages, User.voice_subscribed field, vc_join_voice/vc_leave_voice C ABI
functions, VC_EVENT_VOICE_STATE event, server-side voice_subscribed flag checked
by the SFU relay recipient filter, and core-client gating of remote-stream
decoder setup. All three clients rewired to subscribe+mic on Join / unsubscribe
on Leave. Text chat works regardless of voice subscription.
2. Channel edit dialog now shows the channel's actual current settings. The read
struct vc_channel was missing sort_order and audio fields — only the write
struct vc_channel_info had them. Extended vc_channel with both (additive, no
ABI break), updated the session model and list_channels marshaling to populate
them, and updated all three clients' edit callers to use actual channel info
instead of hardcoded defaults.
3. Channel parameter updates now automatically restart everyone's streams.
Previously editing a channel's audio config persisted and broadcast a
ChannelEvent::UPDATED, but no layer restarted streams — encoders/decoders are
frozen at announce time. handle_channel_event now detects audio-config changes
on the user's current channel and stop->starts each active local stream. The
server reads the updated config on re-announce; peers wire up fresh decoders
at the new ssrc.
All 29 CTest tests pass; Windows DLL + C# client build clean. Apple clients not
yet compile-verified (Windows environment).
2026-06-24 14:29:39 +02:00
|
|
|
CHECK(vc_join_voice(clientB) == VC_OK);
|
|
|
|
|
CHECK(wait_for(evB, [](EventStore& s) { return s.voice_subscribed; }, 5000));
|
|
|
|
|
|
feat(M3): multi-stream & per-channel tuning
Implements docs/roadmap.md M3: multiple concurrent streams per user (MIC +
SCREEN_AUDIO + AUX_DEVICE), independent per-stream receiver gain/mute/noise-
reduction, talk indicators, and enforced per-channel Opus configurability
(mono/stereo, bitrate, frame size, FEC/DTX, application).
Bugs fixed along the way (found while implementing, not pre-existing scope):
- Server hard-coded stream_id=1 for every announce, so a second stream from
the same user silently overwrote the first in SessionRegistry::set_user_stream.
Now a per-session counter (ConnSession::next_stream_id_); handle_stream_stop
validates against announced_stream_ids_ before clearing.
- Client dropped mode/dtx/complexity/application from effective_audio even for
the single M2 stream -- only sample_rate/bitrate_bps/frame_ms/fec were ever
applied to OpusParams. Fixed on both the send (handle_stream_announce_result)
and receive (sync_remote_streams) paths via a shared
opus_params_from_audio_config() helper.
- OpusEncoder always used OPUS_APPLICATION_VOIP; added OpusParams::application
and wired it through.
- on_playback's per-stream decode passed the wrong frame_size to opus_decode
(total samples instead of samples-per-channel), which would have overflowed
the decode buffer for any stereo stream.
- teardown_voice() raced when called concurrently from run_io()'s own cleanup
and from disconnect() on a different thread -- both could see
udp_thread_/talk_timer_thread_ as joinable() at once and race to join() the
same std::thread (intermittent std::system_error under ctest). Fixed with a
teardown_mu_ guard instead of carrying the flake forward.
New:
- Per-channel AudioConfig: SessionRegistry now seeds Lobby (mono/24kbps/VOIP/
FEC+DTX) and a new "Music Room" channel (stereo/128kbps/AUDIO/no DTX);
handle_stream_announce enforces the channel's config, clamping (not
overriding) bitrate_bps to its ceiling.
- core/src/core/client.h/.cpp: local-stream state is now a
std::unordered_map<int, LocalStream> keyed by vc_stream_kind, with
request_id-correlated announce/result handling (request_id already
round-tripped on the wire; just wasn't read before). on_capture_frame is
kind-aware and upmixes mono capture to stereo when a stream's config calls
for it. set_self_mute's mic_muted now only gates the MIC kind. NS is wired
through set_remote_stream. New run_talk_timer() thread emits
VC_EVENT_TALK_STATE from both remote and local edge detection.
- core/src/audio/audio_engine.h/.cpp: kind-keyed injection taps
(inject_capture), stereo-to-mono downmix at the decode/mix boundary,
RemoteStream gains recv_ns (lazy ApmProcessor) + noise_reduction_enabled
and last_voice_ms/talking; new set_stream_noise_reduction() and
poll_talk_transitions().
- core/src/session/session.h/.cpp: Stream now carries the full AudioConfig,
not just sample_rate/frame_ms.
- New additive C ABI (core/include/voicecat.h): vc_audio_config +
vc_get_stream_audio_config (effective Opus config for any stream you own or
a peer's); vc_test_inject_capture (test-only synthetic PCM injection,
clearly marked, mirrors AudioEngine::inject_capture).
- tests/test_m3_multistream.cpp: the M3 exit criterion through the real ABI
(mirrors test_voice_client_abi.cpp's approach, not raw sockets) -- two
concurrent local streams, independent gain/mute/NS control, per-channel
config divergence via vc_get_stream_audio_config, talk indicators.
Explicitly out of scope for this pass (tracked in PROGRESS.md, not silently
dropped): VAD/PTT input gate + device enumeration; real WASAPI loopback
capture for SCREEN_AUDIO (synthetic injection only); true stereo playback
output (AudioEngine's mixer/output device stays mono -- Opus itself is fully
stereo-correct on the wire).
ctest --test-dir build/m1-dev: 11/11 green, verified across 3 consecutive
full-suite runs plus 8 standalone runs of the new test.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 14:12:37 +02:00
|
|
|
uint32_t a_uid = 0;
|
|
|
|
|
{ std::lock_guard lk(evA.mu); a_uid = evA.self_user_id; }
|
|
|
|
|
|
|
|
|
|
// Both guests land in channel 1 (Lobby) automatically; give the async UDP binding
|
|
|
|
|
// handshake a moment to complete on both clients before announcing streams.
|
|
|
|
|
std::this_thread::sleep_for(std::chrono::milliseconds(500));
|
|
|
|
|
|
|
|
|
|
// ── 1. A starts MIC + SCREEN_AUDIO concurrently ──────────────────────────
|
|
|
|
|
vc_stream_desc mic_desc{};
|
|
|
|
|
mic_desc.kind = VC_STREAM_MIC;
|
|
|
|
|
mic_desc.label = "mic";
|
|
|
|
|
uint32_t mic_sid = 0;
|
|
|
|
|
CHECK(vc_stream_start(clientA, &mic_desc, &mic_sid) == VC_OK);
|
|
|
|
|
|
|
|
|
|
vc_stream_desc screen_desc{};
|
|
|
|
|
screen_desc.kind = VC_STREAM_SCREEN_AUDIO;
|
|
|
|
|
screen_desc.label = "desktop audio";
|
|
|
|
|
uint32_t screen_sid = 0;
|
|
|
|
|
CHECK(vc_stream_start(clientA, &screen_desc, &screen_sid) == VC_OK);
|
|
|
|
|
|
|
|
|
|
CHECK(mic_sid != 0 && screen_sid != 0 && mic_sid != screen_sid);
|
|
|
|
|
|
|
|
|
|
// B observes two distinct STREAM_STARTED events for user A.
|
|
|
|
|
bool b_saw_both = wait_for(evB, [&](EventStore& s) {
|
|
|
|
|
bool saw_mic = false, saw_screen = false;
|
|
|
|
|
for (auto& e : s.stream_events) {
|
|
|
|
|
if (!e.started || e.user_id != a_uid) continue;
|
|
|
|
|
if (e.stream_id == mic_sid) saw_mic = true;
|
|
|
|
|
if (e.stream_id == screen_sid) saw_screen = true;
|
|
|
|
|
}
|
|
|
|
|
return saw_mic && saw_screen;
|
|
|
|
|
}, 5000);
|
|
|
|
|
CHECK(b_saw_both);
|
|
|
|
|
|
|
|
|
|
// Also wait for A's own view of both streams (vc_test_inject_capture requires the
|
|
|
|
|
// LocalStream to be active, which flips on A's io_thread_ independently of -- and not
|
|
|
|
|
// necessarily before -- the broadcast B observes above).
|
|
|
|
|
bool a_self_saw_both = wait_for(evA, [&](EventStore& s) {
|
|
|
|
|
bool saw_mic = false, saw_screen = false;
|
|
|
|
|
for (auto& e : s.stream_events) {
|
|
|
|
|
if (!e.started || e.user_id != a_uid) continue;
|
|
|
|
|
if (e.stream_id == mic_sid) saw_mic = true;
|
|
|
|
|
if (e.stream_id == screen_sid) saw_screen = true;
|
|
|
|
|
}
|
|
|
|
|
return saw_mic && saw_screen;
|
|
|
|
|
}, 5000);
|
|
|
|
|
CHECK(a_self_saw_both);
|
|
|
|
|
|
|
|
|
|
// ── 2. Inject synthetic PCM into both of A's local streams ──────────────
|
|
|
|
|
for (int i = 0; i < 25; ++i) {
|
|
|
|
|
auto mic_pcm = make_sine_frame(i, 440.0f);
|
|
|
|
|
auto screen_pcm = make_sine_frame(i, 880.0f);
|
|
|
|
|
CHECK(vc_test_inject_capture(clientA, mic_sid, mic_pcm.data(), mic_pcm.size()) == VC_OK);
|
|
|
|
|
CHECK(vc_test_inject_capture(clientA, screen_sid, screen_pcm.data(), screen_pcm.size()) == VC_OK);
|
|
|
|
|
std::this_thread::sleep_for(std::chrono::milliseconds(20));
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
// No disconnects/errors should have resulted from the dual-stream PCM flow.
|
|
|
|
|
{ std::lock_guard lk(evA.mu); CHECK(!evA.disconnected); }
|
|
|
|
|
{ std::lock_guard lk(evB.mu); CHECK(!evB.disconnected); }
|
|
|
|
|
|
|
|
|
|
// ── 5. Talk indicators ────────────────────────────────────────────────────
|
|
|
|
|
// While A and B are still both in Lobby (voice actually relays between them here --
|
|
|
|
|
// the SFU forwards within a channel, so this must happen before A moves to Music Room
|
|
|
|
|
// in step 4 below), confirm B observed a talking=true edge for A's MIC stream.
|
|
|
|
|
bool b_saw_talking = wait_for(evB, [&](EventStore& s) {
|
|
|
|
|
for (auto& e : s.talk_events)
|
|
|
|
|
if (e.user_id == a_uid && e.stream_id == mic_sid && e.talking) return true;
|
|
|
|
|
return false;
|
|
|
|
|
}, 3000);
|
|
|
|
|
CHECK(b_saw_talking);
|
|
|
|
|
|
|
|
|
|
// ── 3. B independently controls gain/mute/NS on each of A's streams ─────
|
|
|
|
|
CHECK(vc_set_remote_stream(clientB, a_uid, mic_sid, 1.0f, 0, 0) == VC_OK);
|
|
|
|
|
CHECK(vc_set_remote_stream(clientB, a_uid, screen_sid, 0.3f, 1, 1) == VC_OK);
|
|
|
|
|
CHECK(vc_set_remote_stream(clientB, a_uid, 0xDEADBEEF, 1.0f, 0, 0) == VC_ERR_INVALID_ARG);
|
|
|
|
|
|
|
|
|
|
// Toggle NS on/off a few times -- plumbing should never fault or disrupt the stream.
|
|
|
|
|
for (int i = 0; i < 3; ++i) {
|
|
|
|
|
CHECK(vc_set_remote_stream(clientB, a_uid, mic_sid, 1.0f, 0, 1) == VC_OK);
|
|
|
|
|
CHECK(vc_set_remote_stream(clientB, a_uid, mic_sid, 1.0f, 0, 0) == VC_OK);
|
|
|
|
|
}
|
|
|
|
|
{ std::lock_guard lk(evB.mu); CHECK(!evB.disconnected); }
|
|
|
|
|
|
2026-06-18 02:06:44 +02:00
|
|
|
// ── 3b. Read back what B just set (vc_get_remote_stream round-trips the recv state) ─
|
|
|
|
|
{
|
|
|
|
|
vc_remote_stream_state st{};
|
|
|
|
|
// mic_sid: last write above was (1.0, mute=0, nr=0)
|
|
|
|
|
CHECK(vc_get_remote_stream(clientB, a_uid, mic_sid, &st) == VC_OK);
|
|
|
|
|
CHECK(fabsf(st.gain - 1.0f) < 1e-5f);
|
|
|
|
|
CHECK(st.muted == 0);
|
|
|
|
|
CHECK(st.noise_reduction == 0);
|
|
|
|
|
|
|
|
|
|
// screen_sid: set to (0.3, mute=1, nr=1) at line ~282
|
|
|
|
|
CHECK(vc_get_remote_stream(clientB, a_uid, screen_sid, &st) == VC_OK);
|
|
|
|
|
CHECK(fabsf(st.gain - 0.3f) < 1e-5f);
|
|
|
|
|
CHECK(st.muted == 1);
|
|
|
|
|
CHECK(st.noise_reduction == 1);
|
|
|
|
|
|
|
|
|
|
// Unknown stream_id on a known user -> INVALID_ARG.
|
|
|
|
|
CHECK(vc_get_remote_stream(clientB, a_uid, 0xDEADBEEF, &st) == VC_ERR_INVALID_ARG);
|
|
|
|
|
// Null out -> INVALID_ARG.
|
|
|
|
|
CHECK(vc_get_remote_stream(clientB, a_uid, mic_sid, nullptr) == VC_ERR_INVALID_ARG);
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
|
feat(M3): multi-stream & per-channel tuning
Implements docs/roadmap.md M3: multiple concurrent streams per user (MIC +
SCREEN_AUDIO + AUX_DEVICE), independent per-stream receiver gain/mute/noise-
reduction, talk indicators, and enforced per-channel Opus configurability
(mono/stereo, bitrate, frame size, FEC/DTX, application).
Bugs fixed along the way (found while implementing, not pre-existing scope):
- Server hard-coded stream_id=1 for every announce, so a second stream from
the same user silently overwrote the first in SessionRegistry::set_user_stream.
Now a per-session counter (ConnSession::next_stream_id_); handle_stream_stop
validates against announced_stream_ids_ before clearing.
- Client dropped mode/dtx/complexity/application from effective_audio even for
the single M2 stream -- only sample_rate/bitrate_bps/frame_ms/fec were ever
applied to OpusParams. Fixed on both the send (handle_stream_announce_result)
and receive (sync_remote_streams) paths via a shared
opus_params_from_audio_config() helper.
- OpusEncoder always used OPUS_APPLICATION_VOIP; added OpusParams::application
and wired it through.
- on_playback's per-stream decode passed the wrong frame_size to opus_decode
(total samples instead of samples-per-channel), which would have overflowed
the decode buffer for any stereo stream.
- teardown_voice() raced when called concurrently from run_io()'s own cleanup
and from disconnect() on a different thread -- both could see
udp_thread_/talk_timer_thread_ as joinable() at once and race to join() the
same std::thread (intermittent std::system_error under ctest). Fixed with a
teardown_mu_ guard instead of carrying the flake forward.
New:
- Per-channel AudioConfig: SessionRegistry now seeds Lobby (mono/24kbps/VOIP/
FEC+DTX) and a new "Music Room" channel (stereo/128kbps/AUDIO/no DTX);
handle_stream_announce enforces the channel's config, clamping (not
overriding) bitrate_bps to its ceiling.
- core/src/core/client.h/.cpp: local-stream state is now a
std::unordered_map<int, LocalStream> keyed by vc_stream_kind, with
request_id-correlated announce/result handling (request_id already
round-tripped on the wire; just wasn't read before). on_capture_frame is
kind-aware and upmixes mono capture to stereo when a stream's config calls
for it. set_self_mute's mic_muted now only gates the MIC kind. NS is wired
through set_remote_stream. New run_talk_timer() thread emits
VC_EVENT_TALK_STATE from both remote and local edge detection.
- core/src/audio/audio_engine.h/.cpp: kind-keyed injection taps
(inject_capture), stereo-to-mono downmix at the decode/mix boundary,
RemoteStream gains recv_ns (lazy ApmProcessor) + noise_reduction_enabled
and last_voice_ms/talking; new set_stream_noise_reduction() and
poll_talk_transitions().
- core/src/session/session.h/.cpp: Stream now carries the full AudioConfig,
not just sample_rate/frame_ms.
- New additive C ABI (core/include/voicecat.h): vc_audio_config +
vc_get_stream_audio_config (effective Opus config for any stream you own or
a peer's); vc_test_inject_capture (test-only synthetic PCM injection,
clearly marked, mirrors AudioEngine::inject_capture).
- tests/test_m3_multistream.cpp: the M3 exit criterion through the real ABI
(mirrors test_voice_client_abi.cpp's approach, not raw sockets) -- two
concurrent local streams, independent gain/mute/NS control, per-channel
config divergence via vc_get_stream_audio_config, talk indicators.
Explicitly out of scope for this pass (tracked in PROGRESS.md, not silently
dropped): VAD/PTT input gate + device enumeration; real WASAPI loopback
capture for SCREEN_AUDIO (synthetic injection only); true stereo playback
output (AudioEngine's mixer/output device stays mono -- Opus itself is fully
stereo-correct on the wire).
ctest --test-dir build/m1-dev: 11/11 green, verified across 3 consecutive
full-suite runs plus 8 standalone runs of the new test.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 14:12:37 +02:00
|
|
|
// ── 4. Per-channel Opus configurability ──────────────────────────────────
|
|
|
|
|
// A moves to "Music Room" (channel 2: stereo/128kbps/OPUS_AUDIO/no DTX) and announces a
|
|
|
|
|
// fresh MIC stream there; B stays in "Lobby" (channel 1: mono/24kbps/OPUS_VOIP/DTX) with
|
|
|
|
|
// its own MIC stream. Their effective_audio should differ exactly as configured server-side.
|
|
|
|
|
CHECK(vc_stream_stop(clientA, mic_sid) == VC_OK);
|
|
|
|
|
CHECK(vc_join_channel(clientA, 2, nullptr) == VC_OK);
|
|
|
|
|
std::this_thread::sleep_for(std::chrono::milliseconds(300));
|
|
|
|
|
|
|
|
|
|
vc_stream_desc music_mic_desc{};
|
|
|
|
|
music_mic_desc.kind = VC_STREAM_MIC;
|
|
|
|
|
music_mic_desc.label = "music-mic";
|
|
|
|
|
uint32_t a_music_mic_sid = 0;
|
|
|
|
|
CHECK(vc_stream_start(clientA, &music_mic_desc, &a_music_mic_sid) == VC_OK);
|
|
|
|
|
CHECK(wait_for(evA, [&](EventStore& s) {
|
|
|
|
|
for (auto& e : s.stream_events)
|
|
|
|
|
if (e.started && e.user_id == a_uid && e.stream_id == a_music_mic_sid) return true;
|
|
|
|
|
return false;
|
|
|
|
|
}, 5000));
|
|
|
|
|
|
|
|
|
|
vc_stream_desc b_mic_desc{};
|
|
|
|
|
b_mic_desc.kind = VC_STREAM_MIC;
|
|
|
|
|
b_mic_desc.label = "lobby-mic";
|
|
|
|
|
uint32_t b_mic_sid = 0;
|
|
|
|
|
CHECK(vc_stream_start(clientB, &b_mic_desc, &b_mic_sid) == VC_OK);
|
|
|
|
|
CHECK(wait_for(evB, [&](EventStore& s) {
|
|
|
|
|
uint32_t self = s.self_user_id;
|
|
|
|
|
for (auto& e : s.stream_events)
|
|
|
|
|
if (e.started && e.user_id == self && e.stream_id == b_mic_sid) return true;
|
|
|
|
|
return false;
|
|
|
|
|
}, 5000));
|
|
|
|
|
|
|
|
|
|
vc_audio_config a_cfg{};
|
|
|
|
|
vc_audio_config b_cfg{};
|
|
|
|
|
CHECK(vc_get_stream_audio_config(clientA, a_uid, a_music_mic_sid, &a_cfg) == VC_OK);
|
|
|
|
|
uint32_t b_uid = 0;
|
|
|
|
|
{ std::lock_guard lk(evB.mu); b_uid = evB.self_user_id; }
|
|
|
|
|
CHECK(vc_get_stream_audio_config(clientB, b_uid, b_mic_sid, &b_cfg) == VC_OK);
|
|
|
|
|
|
|
|
|
|
// Music Room: stereo, 128kbps, OPUS_AUDIO, DTX off. Lobby: mono, 24kbps, OPUS_VOIP, DTX on.
|
|
|
|
|
CHECK(a_cfg.mode == 1 /* stereo */);
|
|
|
|
|
CHECK(b_cfg.mode == 0 /* mono */);
|
|
|
|
|
CHECK(a_cfg.bitrate_bps == 128000);
|
|
|
|
|
CHECK(b_cfg.bitrate_bps == 24000);
|
|
|
|
|
CHECK(a_cfg.application == 1 /* OPUS_AUDIO */);
|
|
|
|
|
CHECK(b_cfg.application == 0 /* OPUS_VOIP */);
|
|
|
|
|
CHECK(a_cfg.dtx == 0);
|
|
|
|
|
CHECK(b_cfg.dtx != 0);
|
|
|
|
|
|
|
|
|
|
// ── Cleanup ───────────────────────────────────────────────────────────────
|
|
|
|
|
vc_disconnect(clientA);
|
|
|
|
|
vc_disconnect(clientB);
|
|
|
|
|
vc_client_destroy(clientA);
|
|
|
|
|
vc_client_destroy(clientB);
|
|
|
|
|
|
|
|
|
|
server.stop();
|
|
|
|
|
server_thread.join();
|
|
|
|
|
|
|
|
|
|
std::filesystem::remove_all(tmp);
|
|
|
|
|
|
|
|
|
|
if (g_failures == 0) {
|
|
|
|
|
std::printf("m3_multistream: all checks passed\n");
|
|
|
|
|
return 0;
|
|
|
|
|
}
|
|
|
|
|
std::printf("m3_multistream: %d failure(s)\n", g_failures);
|
|
|
|
|
return 1;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#else // !VOICECAT_HAS_NET
|
|
|
|
|
|
|
|
|
|
int main() {
|
|
|
|
|
std::printf("m3_multistream: SKIP (VOICECAT_HAS_NET not defined)\n");
|
|
|
|
|
return 0;
|
|
|
|
|
}
|
|
|
|
|
|
|
|
|
|
#endif // VOICECAT_HAS_NET
|