Files
voice-cat/core/src/session/session.cpp

135 lines
4.9 KiB
C++
Raw Normal View History

#include "session/session.h"
2026-06-15 23:48:44 +02:00
#include <algorithm>
namespace voicecat::session {
2026-06-15 23:48:44 +02:00
const Channel* SessionModel::find_channel(uint32_t id) const {
for (auto& ch : channels_) if (ch.id == id) return &ch;
return nullptr;
}
const User* SessionModel::find_user(uint32_t id) const {
for (auto& u : users_) if (u.id == id) return &u;
return nullptr;
}
std::pair<const User*, const Stream*> SessionModel::find_user_by_ssrc(uint32_t ssrc) const {
for (auto& u : users_) {
for (auto& s : u.streams) {
if (s.ssrc == ssrc) return {&u, &s};
}
}
return {nullptr, nullptr};
}
2026-06-15 23:48:44 +02:00
#ifdef VOICECAT_HAS_NET
namespace {
std::vector<Stream> copy_streams(
const google::protobuf::RepeatedPtrField<voicecat::v1::StreamInfo>& src) {
std::vector<Stream> out;
out.reserve(src.size());
for (const auto& pb : src) {
Stream s;
s.stream_id = pb.stream_id();
s.ssrc = pb.ssrc();
s.kind = static_cast<int>(pb.kind());
s.label = pb.label();
s.sample_rate = pb.audio().sample_rate() ? pb.audio().sample_rate() : 48000;
s.frame_ms = pb.audio().frame_ms() ? pb.audio().frame_ms() : 20;
feat(M3): multi-stream & per-channel tuning Implements docs/roadmap.md M3: multiple concurrent streams per user (MIC + SCREEN_AUDIO + AUX_DEVICE), independent per-stream receiver gain/mute/noise- reduction, talk indicators, and enforced per-channel Opus configurability (mono/stereo, bitrate, frame size, FEC/DTX, application). Bugs fixed along the way (found while implementing, not pre-existing scope): - Server hard-coded stream_id=1 for every announce, so a second stream from the same user silently overwrote the first in SessionRegistry::set_user_stream. Now a per-session counter (ConnSession::next_stream_id_); handle_stream_stop validates against announced_stream_ids_ before clearing. - Client dropped mode/dtx/complexity/application from effective_audio even for the single M2 stream -- only sample_rate/bitrate_bps/frame_ms/fec were ever applied to OpusParams. Fixed on both the send (handle_stream_announce_result) and receive (sync_remote_streams) paths via a shared opus_params_from_audio_config() helper. - OpusEncoder always used OPUS_APPLICATION_VOIP; added OpusParams::application and wired it through. - on_playback's per-stream decode passed the wrong frame_size to opus_decode (total samples instead of samples-per-channel), which would have overflowed the decode buffer for any stereo stream. - teardown_voice() raced when called concurrently from run_io()'s own cleanup and from disconnect() on a different thread -- both could see udp_thread_/talk_timer_thread_ as joinable() at once and race to join() the same std::thread (intermittent std::system_error under ctest). Fixed with a teardown_mu_ guard instead of carrying the flake forward. New: - Per-channel AudioConfig: SessionRegistry now seeds Lobby (mono/24kbps/VOIP/ FEC+DTX) and a new "Music Room" channel (stereo/128kbps/AUDIO/no DTX); handle_stream_announce enforces the channel's config, clamping (not overriding) bitrate_bps to its ceiling. - core/src/core/client.h/.cpp: local-stream state is now a std::unordered_map<int, LocalStream> keyed by vc_stream_kind, with request_id-correlated announce/result handling (request_id already round-tripped on the wire; just wasn't read before). on_capture_frame is kind-aware and upmixes mono capture to stereo when a stream's config calls for it. set_self_mute's mic_muted now only gates the MIC kind. NS is wired through set_remote_stream. New run_talk_timer() thread emits VC_EVENT_TALK_STATE from both remote and local edge detection. - core/src/audio/audio_engine.h/.cpp: kind-keyed injection taps (inject_capture), stereo-to-mono downmix at the decode/mix boundary, RemoteStream gains recv_ns (lazy ApmProcessor) + noise_reduction_enabled and last_voice_ms/talking; new set_stream_noise_reduction() and poll_talk_transitions(). - core/src/session/session.h/.cpp: Stream now carries the full AudioConfig, not just sample_rate/frame_ms. - New additive C ABI (core/include/voicecat.h): vc_audio_config + vc_get_stream_audio_config (effective Opus config for any stream you own or a peer's); vc_test_inject_capture (test-only synthetic PCM injection, clearly marked, mirrors AudioEngine::inject_capture). - tests/test_m3_multistream.cpp: the M3 exit criterion through the real ABI (mirrors test_voice_client_abi.cpp's approach, not raw sockets) -- two concurrent local streams, independent gain/mute/NS control, per-channel config divergence via vc_get_stream_audio_config, talk indicators. Explicitly out of scope for this pass (tracked in PROGRESS.md, not silently dropped): VAD/PTT input gate + device enumeration; real WASAPI loopback capture for SCREEN_AUDIO (synthetic injection only); true stereo playback output (AudioEngine's mixer/output device stays mono -- Opus itself is fully stereo-correct on the wire). ctest --test-dir build/m1-dev: 11/11 green, verified across 3 consecutive full-suite runs plus 8 standalone runs of the new test. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-16 14:12:37 +02:00
s.mode = static_cast<uint32_t>(pb.audio().mode());
s.bitrate_bps = pb.audio().bitrate_bps();
s.application = static_cast<uint32_t>(pb.audio().application());
s.fec = pb.audio().fec();
s.expected_packet_loss = pb.audio().expected_packet_loss();
s.dtx = pb.audio().dtx();
s.complexity = pb.audio().complexity();
out.push_back(std::move(s));
}
return out;
}
} // namespace
2026-06-15 23:48:44 +02:00
void SessionModel::apply_snapshot(const voicecat::v1::ServerStateSnapshot& snap) {
channels_.clear();
for (const auto& pb : snap.channels()) {
Channel ch;
feat(M4): Windows WinForms client, TOFU identity pinning, VAD threshold + always-on mode Core ABI extensions (voicecat.h): - vc_list_channels / vc_list_users / vc_list_user_streams — pull-based snapshot getters for the channel-tree and user-list UI; session_model_mu_ guards cross-thread reads - VC_EVENT_JOIN_RESULT / vc_join_channel — channel join with optional password - VC_EVENT_SERVER_IDENTITY + vc_confirm_server_identity — TOFU gate that blocks io_thread_ until the UI approves or rejects; pins TLS leaf-cert SHA-256 (not declared Ed25519) - vc_get_server_identity_display — Ed25519 fingerprint for human-readable display only - VC_INPUT_ALWAYS_ON = 2 in vc_input_mode — transmit unconditionally, no VAD gate - vc_set_vad_threshold — live RMS threshold update (0.0–1.0); EnergyVadProcessor stores it atomically so the audio RT path reads without a lock C++ implementation: - SessionModel::apply_snapshot / apply_channel_event fixed to populate parent_id, password_protected, and max_users (were permanently zeroed) - TlsContext::peer_cert_fingerprint — SHA-256 of peer leaf cert DER via mbedTLS - TofuStore split into peek (read-only) + pin (write) so first-connect only persists after user approval; tofu_store_path in vc_config for per-user pin file location - TcpAcceptor uses dual-stack IPv6+IPv4 fallback (fixes localhost → ::1 on Windows) - windows-client CMake preset: Release shared DLL, static MinGW runtime, no tools/tests - New C++ tests: test_channel_user_list_abi, test_tofu_flow (14/14 green) Windows client (clients/windows/ — .NET 10 WinForms): - VoiceCat.Interop: LibraryImport P/Invoke surface, UnmanagedCallersOnly callbacks, Channel<VoiceCatEvent> event delivery drained by 30ms WinForms Timer - VoiceCat.App: ConnectDialog (saved servers, DPAPI password storage), ServerIdentity- Dialog (TOFU first-connect / mismatch warning), MainForm (channel TreeView, user ListBox, RichTextBox chat, voice controls, device pickers, VAD/PTT/always-on mode, per-user gain/mute/NR tuning, VAD sensitivity TrackBar, level meter ProgressBar) - PttKeyCaptureDialog — focus-scoped PTT key capture (documented limitation) - PerUserTuningDialog — real-time gain/mute/NR applied to all of a user's streams - Accessibility: explicit AccessibleName/Description on every control, & mnemonics, Activity log ListBox as durable screen-reader record, AutomationNotification for curated live announcements Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:35:16 +02:00
ch.id = pb.id();
ch.parent_id = pb.parent_id();
ch.name = pb.name();
ch.password_protected = pb.password_protected();
ch.max_users = pb.max_users();
2026-06-15 23:48:44 +02:00
channels_.push_back(std::move(ch));
}
users_.clear();
for (const auto& pb : snap.users()) {
User u;
u.id = pb.id();
u.nickname = pb.nickname();
u.is_guest = pb.is_guest();
u.channel_id = pb.channel_id();
u.streams = copy_streams(pb.streams());
2026-06-15 23:48:44 +02:00
users_.push_back(std::move(u));
}
}
void SessionModel::apply_user_event(const voicecat::v1::UserEvent& ev) {
using Kind = voicecat::v1::UserEvent;
if (ev.kind() == Kind::JOINED || ev.kind() == Kind::UPDATED) {
const auto& pb = ev.user();
User u;
u.id = pb.id();
u.nickname = pb.nickname();
u.is_guest = pb.is_guest();
u.channel_id = pb.channel_id();
u.streams = copy_streams(pb.streams());
2026-06-15 23:48:44 +02:00
auto it = std::find_if(users_.begin(), users_.end(),
[&](const User& x) { return x.id == u.id; });
if (it != users_.end()) *it = std::move(u);
else users_.push_back(std::move(u));
} else if (ev.kind() == Kind::LEFT) {
uint32_t uid = ev.user().id();
users_.erase(std::remove_if(users_.begin(), users_.end(),
[uid](const User& x) { return x.id == uid; }),
users_.end());
}
}
void SessionModel::apply_channel_event(const voicecat::v1::ChannelEvent& ev) {
using Kind = voicecat::v1::ChannelEvent;
if (ev.kind() == Kind::CREATED || ev.kind() == Kind::UPDATED) {
const auto& pb = ev.channel();
Channel ch;
feat(M4): Windows WinForms client, TOFU identity pinning, VAD threshold + always-on mode Core ABI extensions (voicecat.h): - vc_list_channels / vc_list_users / vc_list_user_streams — pull-based snapshot getters for the channel-tree and user-list UI; session_model_mu_ guards cross-thread reads - VC_EVENT_JOIN_RESULT / vc_join_channel — channel join with optional password - VC_EVENT_SERVER_IDENTITY + vc_confirm_server_identity — TOFU gate that blocks io_thread_ until the UI approves or rejects; pins TLS leaf-cert SHA-256 (not declared Ed25519) - vc_get_server_identity_display — Ed25519 fingerprint for human-readable display only - VC_INPUT_ALWAYS_ON = 2 in vc_input_mode — transmit unconditionally, no VAD gate - vc_set_vad_threshold — live RMS threshold update (0.0–1.0); EnergyVadProcessor stores it atomically so the audio RT path reads without a lock C++ implementation: - SessionModel::apply_snapshot / apply_channel_event fixed to populate parent_id, password_protected, and max_users (were permanently zeroed) - TlsContext::peer_cert_fingerprint — SHA-256 of peer leaf cert DER via mbedTLS - TofuStore split into peek (read-only) + pin (write) so first-connect only persists after user approval; tofu_store_path in vc_config for per-user pin file location - TcpAcceptor uses dual-stack IPv6+IPv4 fallback (fixes localhost → ::1 on Windows) - windows-client CMake preset: Release shared DLL, static MinGW runtime, no tools/tests - New C++ tests: test_channel_user_list_abi, test_tofu_flow (14/14 green) Windows client (clients/windows/ — .NET 10 WinForms): - VoiceCat.Interop: LibraryImport P/Invoke surface, UnmanagedCallersOnly callbacks, Channel<VoiceCatEvent> event delivery drained by 30ms WinForms Timer - VoiceCat.App: ConnectDialog (saved servers, DPAPI password storage), ServerIdentity- Dialog (TOFU first-connect / mismatch warning), MainForm (channel TreeView, user ListBox, RichTextBox chat, voice controls, device pickers, VAD/PTT/always-on mode, per-user gain/mute/NR tuning, VAD sensitivity TrackBar, level meter ProgressBar) - PttKeyCaptureDialog — focus-scoped PTT key capture (documented limitation) - PerUserTuningDialog — real-time gain/mute/NR applied to all of a user's streams - Accessibility: explicit AccessibleName/Description on every control, & mnemonics, Activity log ListBox as durable screen-reader record, AutomationNotification for curated live announcements Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:35:16 +02:00
ch.id = pb.id();
ch.parent_id = pb.parent_id();
ch.name = pb.name();
ch.password_protected = pb.password_protected();
ch.max_users = pb.max_users();
2026-06-15 23:48:44 +02:00
auto it = std::find_if(channels_.begin(), channels_.end(),
[&](const Channel& x) { return x.id == ch.id; });
if (it != channels_.end()) *it = std::move(ch);
else channels_.push_back(std::move(ch));
} else if (ev.kind() == Kind::DELETED) {
feat(M4): Windows WinForms client, TOFU identity pinning, VAD threshold + always-on mode Core ABI extensions (voicecat.h): - vc_list_channels / vc_list_users / vc_list_user_streams — pull-based snapshot getters for the channel-tree and user-list UI; session_model_mu_ guards cross-thread reads - VC_EVENT_JOIN_RESULT / vc_join_channel — channel join with optional password - VC_EVENT_SERVER_IDENTITY + vc_confirm_server_identity — TOFU gate that blocks io_thread_ until the UI approves or rejects; pins TLS leaf-cert SHA-256 (not declared Ed25519) - vc_get_server_identity_display — Ed25519 fingerprint for human-readable display only - VC_INPUT_ALWAYS_ON = 2 in vc_input_mode — transmit unconditionally, no VAD gate - vc_set_vad_threshold — live RMS threshold update (0.0–1.0); EnergyVadProcessor stores it atomically so the audio RT path reads without a lock C++ implementation: - SessionModel::apply_snapshot / apply_channel_event fixed to populate parent_id, password_protected, and max_users (were permanently zeroed) - TlsContext::peer_cert_fingerprint — SHA-256 of peer leaf cert DER via mbedTLS - TofuStore split into peek (read-only) + pin (write) so first-connect only persists after user approval; tofu_store_path in vc_config for per-user pin file location - TcpAcceptor uses dual-stack IPv6+IPv4 fallback (fixes localhost → ::1 on Windows) - windows-client CMake preset: Release shared DLL, static MinGW runtime, no tools/tests - New C++ tests: test_channel_user_list_abi, test_tofu_flow (14/14 green) Windows client (clients/windows/ — .NET 10 WinForms): - VoiceCat.Interop: LibraryImport P/Invoke surface, UnmanagedCallersOnly callbacks, Channel<VoiceCatEvent> event delivery drained by 30ms WinForms Timer - VoiceCat.App: ConnectDialog (saved servers, DPAPI password storage), ServerIdentity- Dialog (TOFU first-connect / mismatch warning), MainForm (channel TreeView, user ListBox, RichTextBox chat, voice controls, device pickers, VAD/PTT/always-on mode, per-user gain/mute/NR tuning, VAD sensitivity TrackBar, level meter ProgressBar) - PttKeyCaptureDialog — focus-scoped PTT key capture (documented limitation) - PerUserTuningDialog — real-time gain/mute/NR applied to all of a user's streams - Accessibility: explicit AccessibleName/Description on every control, & mnemonics, Activity log ListBox as durable screen-reader record, AutomationNotification for curated live announcements Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-17 00:35:16 +02:00
// deleted_id, not channel().id() — the proto leaves `channel` unset for deletes
// (docs/protocol.md, core/proto/voicecat.proto's ChannelEvent). Pre-existing bug, dead
// code until something actually emits ChannelEvent (no channel CRUD exists yet — M5+),
// fixed here while touching this function for the M4 field-population fix.
uint32_t cid = ev.deleted_id();
2026-06-15 23:48:44 +02:00
channels_.erase(std::remove_if(channels_.begin(), channels_.end(),
[cid](const Channel& x) { return x.id == cid; }),
channels_.end());
}
}
#endif // VOICECAT_HAS_NET
} // namespace voicecat::session