feat(M3): multi-stream & per-channel tuning
Implements docs/roadmap.md M3: multiple concurrent streams per user (MIC + SCREEN_AUDIO + AUX_DEVICE), independent per-stream receiver gain/mute/noise- reduction, talk indicators, and enforced per-channel Opus configurability (mono/stereo, bitrate, frame size, FEC/DTX, application). Bugs fixed along the way (found while implementing, not pre-existing scope): - Server hard-coded stream_id=1 for every announce, so a second stream from the same user silently overwrote the first in SessionRegistry::set_user_stream. Now a per-session counter (ConnSession::next_stream_id_); handle_stream_stop validates against announced_stream_ids_ before clearing. - Client dropped mode/dtx/complexity/application from effective_audio even for the single M2 stream -- only sample_rate/bitrate_bps/frame_ms/fec were ever applied to OpusParams. Fixed on both the send (handle_stream_announce_result) and receive (sync_remote_streams) paths via a shared opus_params_from_audio_config() helper. - OpusEncoder always used OPUS_APPLICATION_VOIP; added OpusParams::application and wired it through. - on_playback's per-stream decode passed the wrong frame_size to opus_decode (total samples instead of samples-per-channel), which would have overflowed the decode buffer for any stereo stream. - teardown_voice() raced when called concurrently from run_io()'s own cleanup and from disconnect() on a different thread -- both could see udp_thread_/talk_timer_thread_ as joinable() at once and race to join() the same std::thread (intermittent std::system_error under ctest). Fixed with a teardown_mu_ guard instead of carrying the flake forward. New: - Per-channel AudioConfig: SessionRegistry now seeds Lobby (mono/24kbps/VOIP/ FEC+DTX) and a new "Music Room" channel (stereo/128kbps/AUDIO/no DTX); handle_stream_announce enforces the channel's config, clamping (not overriding) bitrate_bps to its ceiling. - core/src/core/client.h/.cpp: local-stream state is now a std::unordered_map<int, LocalStream> keyed by vc_stream_kind, with request_id-correlated announce/result handling (request_id already round-tripped on the wire; just wasn't read before). on_capture_frame is kind-aware and upmixes mono capture to stereo when a stream's config calls for it. set_self_mute's mic_muted now only gates the MIC kind. NS is wired through set_remote_stream. New run_talk_timer() thread emits VC_EVENT_TALK_STATE from both remote and local edge detection. - core/src/audio/audio_engine.h/.cpp: kind-keyed injection taps (inject_capture), stereo-to-mono downmix at the decode/mix boundary, RemoteStream gains recv_ns (lazy ApmProcessor) + noise_reduction_enabled and last_voice_ms/talking; new set_stream_noise_reduction() and poll_talk_transitions(). - core/src/session/session.h/.cpp: Stream now carries the full AudioConfig, not just sample_rate/frame_ms. - New additive C ABI (core/include/voicecat.h): vc_audio_config + vc_get_stream_audio_config (effective Opus config for any stream you own or a peer's); vc_test_inject_capture (test-only synthetic PCM injection, clearly marked, mirrors AudioEngine::inject_capture). - tests/test_m3_multistream.cpp: the M3 exit criterion through the real ABI (mirrors test_voice_client_abi.cpp's approach, not raw sockets) -- two concurrent local streams, independent gain/mute/NS control, per-channel config divergence via vc_get_stream_audio_config, talk indicators. Explicitly out of scope for this pass (tracked in PROGRESS.md, not silently dropped): VAD/PTT input gate + device enumeration; real WASAPI loopback capture for SCREEN_AUDIO (synthetic injection only); true stereo playback output (AudioEngine's mixer/output device stays mono -- Opus itself is fully stereo-correct on the wire). ctest --test-dir build/m1-dev: 11/11 green, verified across 3 consecutive full-suite runs plus 8 standalone runs of the new test. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
137
PROGRESS.md
137
PROGRESS.md
@@ -10,25 +10,20 @@ up instantly. Newest status at the top.
|
|||||||
|
|
||||||
## ▶ Where we left off / next action
|
## ▶ Where we left off / next action
|
||||||
|
|
||||||
- **Done:** **M2 — voice, single stream** ✓ complete (2026-06-16), now genuinely satisfied
|
- **Done:** **M3 — multi-stream & per-channel tuning** ✓ complete (2026-06-16). See the M3
|
||||||
through the real client library, not just `test_m2_voice`'s raw-socket harness.
|
section below for the full file-by-file change list. `ctest --test-dir build/m1-dev` —
|
||||||
`vc_stream_start/stop`, UDP binding, capture→encode→seal→send, and recv→open→decode→
|
**11/11 tests** green (3 consecutive full-suite runs), including the new
|
||||||
playback were all `VC_ERR_NOT_IMPLEMENTED` stubs in `core/src/core/client.cpp` even after
|
`test_m3_multistream` (real `vc_client`s, not raw sockets — same lesson as M2: ABI-level
|
||||||
`test_m2_voice` went green — meaning `vccli`/any GUI client still couldn't actually talk.
|
coverage is what proves the client library, not just the wire protocol).
|
||||||
Implemented for real this session, plus server-side `StreamInfo` broadcast
|
**Two items intentionally still open** (carried forward, not silently dropped):
|
||||||
(`SessionRegistry::set_user_stream/clear_user_stream` → `UserEvent::UPDATED`) so a second
|
- `vc_set_input_device`/`vc_set_input_mode`/`vc_set_push_to_talk`/`vc_list_devices`
|
||||||
client's `sync_remote_streams()` learns about a peer's stream without polling.
|
(device enumeration + VAD/PTT input gate) remain `VC_ERR_NOT_IMPLEMENTED` — explicitly
|
||||||
`ctest --test-dir build/m1-dev` — **10/10 tests** green, including the new
|
scoped out of this M3 pass; revisit in a future milestone.
|
||||||
`test_voice_client_abi` (two real `vc_client` instances, not raw sockets, drive the full
|
- Stereo Opus is now wire-correct end-to-end (a channel configured `MODE_STEREO` really
|
||||||
UDP-binding → stream-announce → cross-client `STREAM_STARTED`/`STOPPED` event path).
|
encodes/decodes 2-channel Opus packets), but `AudioEngine`'s playback mixer/output device
|
||||||
`vccli --voice` (new flag, alongside `--host/--port/--nick/--channel/--mute/--text`) was
|
stays mono internally — stereo streams are downmixed (avg L/R) immediately after decode,
|
||||||
manually verified live: two instances see each other's mic-stream start/stop in real time.
|
before mixing. True stereo *playback output* is a follow-up, not part of M3.
|
||||||
Note: `test_m1_integration` and `test_m2_voice` have a pre-existing intermittent flake on
|
- **Next:** **M4 — native clients** (Windows C#, macOS/iOS Swift). See `docs/roadmap.md §M4`.
|
||||||
Windows in their cleanup paths (thread-join / SQLite-file-handle release race, unrelated to
|
|
||||||
this session's changes) — rerun in isolation if one fails standalone in the full suite.
|
|
||||||
- **Next:** **M3 — multi-stream & per-channel tuning** (screen audio, listener-side per-user
|
|
||||||
NR, jitter buffer stats API — note `voicecat.h` currently exposes no stats getter beyond
|
|
||||||
`on_level`'s RMS meter, so a stats API needs new C ABI surface). See `docs/roadmap.md §M3`.
|
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -37,9 +32,8 @@ up instantly. Newest status at the top.
|
|||||||
- [x] **M0 — Scaffolding** ✓ complete
|
- [x] **M0 — Scaffolding** ✓ complete
|
||||||
- [x] **M1 — Control plane** ✓ complete (2026-06-15)
|
- [x] **M1 — Control plane** ✓ complete (2026-06-15)
|
||||||
- [x] **M2 — Voice, single stream** ✓ complete (2026-06-16)
|
- [x] **M2 — Voice, single stream** ✓ complete (2026-06-16)
|
||||||
- [ ] **M3 — Multi-stream & per-channel tuning** ← next
|
- [x] **M3 — Multi-stream & per-channel tuning** ✓ complete (2026-06-16)
|
||||||
- [ ] **M3 — Multi-stream & per-channel tuning** (screen audio, listener-side per-user NR)
|
- [ ] **M4 — Native clients** (Windows C#, macOS/iOS Swift) ← next
|
||||||
- [ ] **M4 — Native clients** (Windows C#, macOS/iOS Swift)
|
|
||||||
- [ ] **M5 — Moderation, polish, beyond** (perms, bans, DRED; then file transfer, E2EE, …)
|
- [ ] **M5 — Moderation, polish, beyond** (perms, bans, DRED; then file transfer, E2EE, …)
|
||||||
|
|
||||||
---
|
---
|
||||||
@@ -140,6 +134,105 @@ wasn't met. Closed the gap:
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## M3 — Multi-stream & per-channel tuning ✓ (completed 2026-06-16)
|
||||||
|
|
||||||
|
**Exit criterion:** ✓ `test_m3_multistream` — a real `vc_client` (A) runs two concurrent local
|
||||||
|
streams (MIC + SCREEN_AUDIO) with distinct stream ids; a second client (B) sees both as
|
||||||
|
separate `STREAM_STARTED` events and a `VC_EVENT_TALK_STATE` talking edge for A's MIC stream;
|
||||||
|
B independently sets gain/mute/noise-reduction on each of A's streams without one call
|
||||||
|
affecting the other; A then joins "Music Room" (channel 2: stereo/128kbps/`OPUS_AUDIO`/no
|
||||||
|
DTX) and announces a fresh MIC stream there, while B stays in "Lobby" (channel 1: mono/24kbps/
|
||||||
|
`OPUS_VOIP`/DTX on) — `vc_get_stream_audio_config` shows their effective Opus config differs
|
||||||
|
exactly as the server enforces per channel. Passes in ~2.4s; verified across 8 consecutive
|
||||||
|
standalone runs + 3 consecutive full-suite runs with no flakes.
|
||||||
|
|
||||||
|
Exploration before implementing turned up several bugs/gaps where the wire format already
|
||||||
|
supported this milestone but the client/server logic didn't — these were fixed as part of M3,
|
||||||
|
not treated as pre-existing-and-out-of-scope:
|
||||||
|
|
||||||
|
- [x] **Server `stream_id` bug** — `handle_stream_announce` always wrote `stream_id=1`, so a
|
||||||
|
second stream from the same user silently overwrote the first in
|
||||||
|
`SessionRegistry::set_user_stream`'s replace-by-id logic. Fixed with a per-session counter
|
||||||
|
(`ConnSession::next_stream_id_`) + `announced_stream_ids_` (also now validated in
|
||||||
|
`handle_stream_stop`, rejecting stops for ids the session never announced).
|
||||||
|
- [x] **Per-channel `AudioConfig` was modeled but never populated/enforced.**
|
||||||
|
`SessionRegistry::init_default_channels()` now seeds Lobby (id=1: mono, 24kbps, `OPUS_VOIP`,
|
||||||
|
FEC+DTX on) and a new "Music Room" (id=2: stereo, 128kbps, `OPUS_AUDIO`, FEC+DTX off) with
|
||||||
|
real `AudioConfig`s; new `SessionRegistry::channel_audio_config(channel_id)` accessor (there
|
||||||
|
was no per-id channel getter before, only `channel_snapshot()`). `handle_stream_announce`
|
||||||
|
now treats the channel's config as authoritative (mode/frame_ms/application/fec/dtx/
|
||||||
|
complexity), clamping (not overriding) `bitrate_bps` to the channel's ceiling.
|
||||||
|
- [x] **Client silently dropped `mode`/`dtx`/`complexity`/`application` from `effective_audio`**
|
||||||
|
even for the single M2 stream — `handle_stream_announce_result` and `sync_remote_streams`
|
||||||
|
only copied `sample_rate`/`bitrate_bps`/`frame_ms`/`fec` into `OpusParams`. New shared
|
||||||
|
`opus_params_from_audio_config()` helper (`client.cpp`) fixes both the send and receive
|
||||||
|
paths.
|
||||||
|
- [x] `core/src/codec/opus_codec.h/.cpp` — new `OpusApplication` enum + `OpusParams::application`
|
||||||
|
field; `OpusEncoder::init` now honors it instead of hardcoding `OPUS_APPLICATION_VOIP`.
|
||||||
|
- [x] `core/src/session/session.h/.cpp` — `Stream` struct extended with the full `AudioConfig`
|
||||||
|
(mode/bitrate_bps/application/fec/expected_packet_loss/dtx/complexity), not just
|
||||||
|
sample_rate/frame_ms; `copy_streams()` now copies all of it.
|
||||||
|
- [x] `core/src/core/client.h/.cpp` — local-stream state is now a `std::unordered_map<int,
|
||||||
|
LocalStream>` keyed by `vc_stream_kind` (one active stream per kind — MIC/SCREEN_AUDIO/
|
||||||
|
AUX_DEVICE are each singletons for a client), replacing the M2 single-stream fields.
|
||||||
|
`StreamAnnounce`/`StreamAnnounceResult` round-trips are now correlated by `request_id`
|
||||||
|
(already round-tripped on the wire; just wasn't read) via `pending_announce_kind_`, so
|
||||||
|
multiple concurrent announces from one client resolve to the right `LocalStream`.
|
||||||
|
`on_capture_frame` takes a `kind` parameter and upmixes mono capture to stereo (duplicate
|
||||||
|
L=R) when a stream's channel config calls for it. `vc_set_self_mute`'s `mic_muted` only
|
||||||
|
gates the `MIC` kind — a concurrent `SCREEN_AUDIO` share keeps playing while muted.
|
||||||
|
`set_remote_stream` now actually wires `noise_reduction` through (previously parsed and
|
||||||
|
discarded). New `run_talk_timer()` (a small dedicated thread, started alongside the UDP
|
||||||
|
media path, never the miniaudio callback thread) polls both remote talk-state edges
|
||||||
|
(`AudioEngine::poll_talk_transitions()`) and local capture-activity edges, emitting
|
||||||
|
`VC_EVENT_TALK_STATE`.
|
||||||
|
- [x] **Fixed a thread-join race in `teardown_voice()`** — it's called both from `run_io()`'s
|
||||||
|
own cleanup and from `disconnect()`, on different threads; without serialization both could
|
||||||
|
see `udp_thread_`/`talk_timer_thread_` as `joinable()` simultaneously and race to `join()`
|
||||||
|
the same `std::thread` (UB; surfaced as an intermittent `std::system_error: No such process`
|
||||||
|
under `ctest`). Added a `teardown_mu_` guard around the whole function. This pre-existed for
|
||||||
|
`udp_thread_` alone (likely the same root cause as the `test_m1_integration`/`test_m2_voice`
|
||||||
|
cleanup-path flake noted in the M2 section above) — adding `talk_timer_thread_`'s join just
|
||||||
|
made it surface more often, so it was fixed properly here rather than carried forward again.
|
||||||
|
- [x] `core/src/audio/audio_engine.h/.cpp` — `CaptureCallback` gained a `kind` parameter
|
||||||
|
(the real miniaudio capture device is always tagged `kind=0`/MIC; a second concurrent local
|
||||||
|
stream is fed via its own `inject_capture(kind, ...)` ring buffer — `inject_taps_`, keyed by
|
||||||
|
kind — since there is only one real hardware capture device in M3). Fixed a buffer-sizing
|
||||||
|
bug in `on_playback`'s per-stream decode (`opus_decode`'s `frame_size` parameter is
|
||||||
|
samples-*per-channel*, not total samples — the old code passed `frames * params_.channels`,
|
||||||
|
which would have overflowed the decode buffer for any stereo stream). Stereo decoder output
|
||||||
|
is downmixed (avg L/R) into the engine's mono mix accumulator immediately after decode.
|
||||||
|
`RemoteStream` gained `recv_ns`/`noise_reduction_enabled` (lazy `ApmProcessor` instantiation
|
||||||
|
— freed on disable, so no separate instance cap is needed per the roadmap's guidance) and
|
||||||
|
`last_voice_ms`/`talking` (talk-indicator edge state, updated in `push_recv_frame`); new
|
||||||
|
`set_stream_noise_reduction()` and `poll_talk_transitions()`. Note: until `VOICECAT_HAS_APM`
|
||||||
|
is wired to a real WebRTC APM build, the NS toggle is plumbed end-to-end but behaviorally a
|
||||||
|
passthrough no-op (`ApmPassthrough` doesn't touch PCM) — same situation send-side APM has
|
||||||
|
been in since M2; M3's job was the plumbing, not the DSP backend.
|
||||||
|
- [x] **New C ABI surface** (`core/include/voicecat.h`, additive only):
|
||||||
|
`vc_audio_config` struct + `vc_get_stream_audio_config(c, user_id, stream_id, out)` — the
|
||||||
|
effective Opus config for a stream you own or a peer's, reading from the (now richer)
|
||||||
|
`LocalStream`/`session::Stream`. `vc_test_inject_capture(c, stream_id, pcm, samples)` —
|
||||||
|
clearly-marked **test-only**, forwards to `AudioEngine::inject_capture`, so
|
||||||
|
`test_m3_multistream` can drive two concurrent synthetic-audio streams through the real ABI
|
||||||
|
without a microphone.
|
||||||
|
- [x] `tests/test_m3_multistream.cpp` — the M3 exit criterion (ABI-level, mirrors
|
||||||
|
`test_voice_client_abi.cpp`'s approach per the M2 lesson). Registered in `tests/CMakeLists.txt`.
|
||||||
|
|
||||||
|
**Explicitly out of scope for this pass** (confirmed with the user before implementing):
|
||||||
|
- `vc_set_input_device`/`vc_set_input_mode`/`vc_set_push_to_talk`/`vc_list_devices` (device
|
||||||
|
enumeration + VAD/PTT input gate) — still `VC_ERR_NOT_IMPLEMENTED`. These were mentioned as
|
||||||
|
"scoped to M3" in the M2 follow-up notes above, but docs/roadmap.md's M3 bullets never
|
||||||
|
actually listed them — deferred again, now tracked explicitly rather than implicitly.
|
||||||
|
- Real WASAPI desktop-audio loopback capture for `SCREEN_AUDIO` — the engine now supports
|
||||||
|
feeding a second concurrent local stream via `inject_capture`, but only synthetic PCM is
|
||||||
|
wired up; a real loopback capture device is a follow-up.
|
||||||
|
- True stereo *playback output* — `AudioEngine`'s mixer/output device stays mono; stereo
|
||||||
|
streams are downmixed after decode (see above). The Opus wire format itself is fully
|
||||||
|
stereo-correct.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Decisions log
|
## Decisions log
|
||||||
|
|
||||||
All architecture/scope decisions are settled and recorded in
|
All architecture/scope decisions are settled and recorded in
|
||||||
|
|||||||
@@ -158,6 +158,22 @@ typedef struct vc_stream_desc {
|
|||||||
const char* label; /* human label, e.g. "Microphone" */
|
const char* label; /* human label, e.g. "Microphone" */
|
||||||
} vc_stream_desc;
|
} vc_stream_desc;
|
||||||
|
|
||||||
|
/* The effective Opus configuration in use for a stream — for a stream you own, this is
|
||||||
|
* StreamAnnounceResult.effective_audio (channel-enforced, docs/voice.md §3); for a remote
|
||||||
|
* stream, it's the peer's broadcast StreamInfo.audio. See vc_get_stream_audio_config. */
|
||||||
|
typedef struct vc_audio_config {
|
||||||
|
uint32_t codec; /* 0 = OPUS */
|
||||||
|
uint32_t mode; /* 0 = mono, 1 = stereo */
|
||||||
|
uint32_t sample_rate;
|
||||||
|
uint32_t bitrate_bps;
|
||||||
|
uint32_t frame_ms;
|
||||||
|
uint32_t application; /* 0 = VOIP, 1 = AUDIO, 2 = LOWDELAY */
|
||||||
|
int fec; /* bool */
|
||||||
|
uint32_t expected_packet_loss; /* % 0..100 */
|
||||||
|
int dtx; /* bool */
|
||||||
|
uint32_t complexity; /* 0..10 */
|
||||||
|
} vc_audio_config;
|
||||||
|
|
||||||
typedef struct vc_device {
|
typedef struct vc_device {
|
||||||
const char* id;
|
const char* id;
|
||||||
const char* name;
|
const char* name;
|
||||||
@@ -208,6 +224,18 @@ VC_API vc_result vc_set_self_mute(vc_client* c, int mic_muted, int deafened);
|
|||||||
VC_API vc_result vc_set_remote_stream(vc_client* c, uint32_t user_id, uint32_t stream_id,
|
VC_API vc_result vc_set_remote_stream(vc_client* c, uint32_t user_id, uint32_t stream_id,
|
||||||
float gain, int muted, int noise_reduction);
|
float gain, int muted, int noise_reduction);
|
||||||
|
|
||||||
|
/* Effective Opus config in use for (user_id, stream_id) — your own stream or a peer's.
|
||||||
|
* VC_ERR_INVALID_ARG if the user/stream isn't known. */
|
||||||
|
VC_API vc_result vc_get_stream_audio_config(vc_client* c, uint32_t user_id, uint32_t stream_id,
|
||||||
|
vc_audio_config* out);
|
||||||
|
|
||||||
|
/* TEST-ONLY — not for production use. Bypasses the real capture device, injecting raw PCM
|
||||||
|
* directly into the named local stream's encode pipeline (see AudioEngine::inject_capture).
|
||||||
|
* Exists so automated tests can drive the real vc_client/ABI path end-to-end without a
|
||||||
|
* microphone. `stream_id` is the id returned by vc_stream_start. */
|
||||||
|
VC_API vc_result vc_test_inject_capture(vc_client* c, uint32_t stream_id, const int16_t* pcm,
|
||||||
|
size_t samples);
|
||||||
|
|
||||||
/* ── Text ─────────────────────────────────────────────────────────────────── */
|
/* ── Text ─────────────────────────────────────────────────────────────────── */
|
||||||
VC_API vc_result vc_send_text(vc_client* c, vc_text_scope scope, uint32_t target_id,
|
VC_API vc_result vc_send_text(vc_client* c, vc_text_scope scope, uint32_t target_id,
|
||||||
const char* utf8);
|
const char* utf8);
|
||||||
|
|||||||
@@ -6,10 +6,19 @@
|
|||||||
#include "audio/audio_engine.h"
|
#include "audio/audio_engine.h"
|
||||||
|
|
||||||
#include <algorithm>
|
#include <algorithm>
|
||||||
|
#include <chrono>
|
||||||
#include <cstring>
|
#include <cstring>
|
||||||
|
|
||||||
namespace voicecat::audio {
|
namespace voicecat::audio {
|
||||||
|
|
||||||
|
namespace {
|
||||||
|
int64_t now_ms() {
|
||||||
|
return std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||||
|
std::chrono::steady_clock::now().time_since_epoch())
|
||||||
|
.count();
|
||||||
|
}
|
||||||
|
} // namespace
|
||||||
|
|
||||||
// ── JitterBuffer ─────────────────────────────────────────────────────────────
|
// ── JitterBuffer ─────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
void JitterBuffer::push(Frame f) {
|
void JitterBuffer::push(Frame f) {
|
||||||
@@ -66,9 +75,7 @@ void JitterBuffer::reset() {
|
|||||||
|
|
||||||
// ── AudioEngine ──────────────────────────────────────────────────────────────
|
// ── AudioEngine ──────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
AudioEngine::AudioEngine() {
|
AudioEngine::AudioEngine() = default;
|
||||||
inject_ring_.resize(kInjectCapSamples, 0);
|
|
||||||
}
|
|
||||||
|
|
||||||
AudioEngine::~AudioEngine() { stop(); }
|
AudioEngine::~AudioEngine() { stop(); }
|
||||||
|
|
||||||
@@ -135,31 +142,43 @@ void AudioEngine::stop() {
|
|||||||
#endif
|
#endif
|
||||||
}
|
}
|
||||||
|
|
||||||
void AudioEngine::inject_capture(const int16_t* pcm, size_t n) {
|
void AudioEngine::inject_capture(int kind, const int16_t* pcm, size_t n) {
|
||||||
|
InjectTap* tap;
|
||||||
|
{
|
||||||
std::lock_guard lk(inject_mu_);
|
std::lock_guard lk(inject_mu_);
|
||||||
size_t w = inject_write_.load(std::memory_order_relaxed);
|
auto& slot = inject_taps_[kind];
|
||||||
|
if (!slot) {
|
||||||
|
slot = std::make_unique<InjectTap>();
|
||||||
|
slot->ring.resize(kInjectCapSamples, 0);
|
||||||
|
}
|
||||||
|
tap = slot.get();
|
||||||
|
}
|
||||||
|
|
||||||
|
size_t w = tap->write.load(std::memory_order_relaxed);
|
||||||
for (size_t i = 0; i < n; ++i)
|
for (size_t i = 0; i < n; ++i)
|
||||||
inject_ring_[(w + i) % kInjectCapSamples] = pcm[i];
|
tap->ring[(w + i) % kInjectCapSamples] = pcm[i];
|
||||||
inject_write_.store(w + n, std::memory_order_release);
|
tap->write.store(w + n, std::memory_order_release);
|
||||||
|
|
||||||
// Fire capture_cb_ for each complete frame now available.
|
// Fire capture_cb_ for each complete frame now available.
|
||||||
while (true) {
|
while (true) {
|
||||||
size_t r = inject_read_.load(std::memory_order_relaxed);
|
size_t r = tap->read.load(std::memory_order_relaxed);
|
||||||
size_t avail = inject_write_.load(std::memory_order_acquire) - r;
|
size_t avail = tap->write.load(std::memory_order_acquire) - r;
|
||||||
if (avail < static_cast<size_t>(frame_samples_)) break;
|
if (avail < static_cast<size_t>(frame_samples_)) break;
|
||||||
|
|
||||||
std::vector<int16_t> frame(frame_samples_);
|
std::vector<int16_t> frame(frame_samples_);
|
||||||
for (int i = 0; i < frame_samples_; ++i)
|
for (int i = 0; i < frame_samples_; ++i)
|
||||||
frame[i] = inject_ring_[(r + i) % kInjectCapSamples];
|
frame[i] = tap->ring[(r + i) % kInjectCapSamples];
|
||||||
inject_read_.store(r + frame_samples_, std::memory_order_release);
|
tap->read.store(r + frame_samples_, std::memory_order_release);
|
||||||
|
|
||||||
if (capture_cb_) capture_cb_(frame.data(), frame_samples_);
|
if (capture_cb_) capture_cb_(kind, frame.data(), frame_samples_);
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
void AudioEngine::push_recv_frame(uint32_t ssrc, JitterBuffer::Frame f) {
|
void AudioEngine::push_recv_frame(uint32_t ssrc, JitterBuffer::Frame f) {
|
||||||
std::lock_guard lk(streams_mu_);
|
std::lock_guard lk(streams_mu_);
|
||||||
streams_[ssrc].jitter.push(std::move(f));
|
auto& s = streams_[ssrc];
|
||||||
|
s.last_voice_ms.store(now_ms(), std::memory_order_relaxed);
|
||||||
|
s.jitter.push(std::move(f));
|
||||||
}
|
}
|
||||||
|
|
||||||
void AudioEngine::set_stream_gain(uint32_t ssrc, float gain) {
|
void AudioEngine::set_stream_gain(uint32_t ssrc, float gain) {
|
||||||
@@ -172,11 +191,37 @@ void AudioEngine::set_stream_mute(uint32_t ssrc, bool mute) {
|
|||||||
streams_[ssrc].mute = mute;
|
streams_[ssrc].mute = mute;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
void AudioEngine::set_stream_noise_reduction(uint32_t ssrc, bool enable) {
|
||||||
|
std::lock_guard lk(streams_mu_);
|
||||||
|
auto& s = streams_[ssrc];
|
||||||
|
s.noise_reduction_enabled = enable;
|
||||||
|
if (enable) {
|
||||||
|
if (!s.recv_ns) s.recv_ns = ApmProcessor::create();
|
||||||
|
} else {
|
||||||
|
s.recv_ns.reset();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
void AudioEngine::remove_stream(uint32_t ssrc) {
|
void AudioEngine::remove_stream(uint32_t ssrc) {
|
||||||
std::lock_guard lk(streams_mu_);
|
std::lock_guard lk(streams_mu_);
|
||||||
streams_.erase(ssrc);
|
streams_.erase(ssrc);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
std::vector<std::pair<uint32_t, bool>> AudioEngine::poll_talk_transitions() {
|
||||||
|
std::vector<std::pair<uint32_t, bool>> edges;
|
||||||
|
std::lock_guard lk(streams_mu_);
|
||||||
|
int64_t now = now_ms();
|
||||||
|
for (auto& [ssrc, stream] : streams_) {
|
||||||
|
bool now_talking = (now - stream.last_voice_ms.load(std::memory_order_relaxed)) <
|
||||||
|
kTalkHangoverMs;
|
||||||
|
if (now_talking != stream.talking) {
|
||||||
|
stream.talking = now_talking;
|
||||||
|
edges.emplace_back(ssrc, now_talking);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
return edges;
|
||||||
|
}
|
||||||
|
|
||||||
uint32_t AudioEngine::stream_packets_lost(uint32_t ssrc) const {
|
uint32_t AudioEngine::stream_packets_lost(uint32_t ssrc) const {
|
||||||
std::lock_guard lk(streams_mu_);
|
std::lock_guard lk(streams_mu_);
|
||||||
auto it = streams_.find(ssrc);
|
auto it = streams_.find(ssrc);
|
||||||
@@ -205,7 +250,10 @@ void AudioEngine::capture_data_cb(ma_device* dev, void* /*out*/,
|
|||||||
}
|
}
|
||||||
|
|
||||||
void AudioEngine::on_capture(const int16_t* pcm, ma_uint32 frames) {
|
void AudioEngine::on_capture(const int16_t* pcm, ma_uint32 frames) {
|
||||||
if (capture_cb_) capture_cb_(pcm, static_cast<int>(frames));
|
// The real hardware capture device is always the "primary" tap (kind 0 / MIC). A second
|
||||||
|
// concurrent local stream (e.g. SCREEN_AUDIO) is fed via inject_capture() in M3 — there is
|
||||||
|
// only one real capture device.
|
||||||
|
if (capture_cb_) capture_cb_(0, pcm, static_cast<int>(frames));
|
||||||
}
|
}
|
||||||
|
|
||||||
void AudioEngine::playback_data_cb(ma_device* dev, void* out,
|
void AudioEngine::playback_data_cb(ma_device* dev, void* out,
|
||||||
@@ -226,24 +274,43 @@ void AudioEngine::on_playback(int16_t* out, ma_uint32 frames) {
|
|||||||
for (auto& [ssrc, stream] : streams_) {
|
for (auto& [ssrc, stream] : streams_) {
|
||||||
if (stream.mute || !stream.decoder.valid()) continue;
|
if (stream.mute || !stream.decoder.valid()) continue;
|
||||||
|
|
||||||
|
// M3: a stream's Opus channel count (mono/stereo, per-channel AudioConfig) may differ
|
||||||
|
// from the engine-wide playback channel count (always mono in M3 — see PROGRESS.md).
|
||||||
|
// Decode into a buffer sized for the *decoder's* channel count (opus_decode's
|
||||||
|
// frame_size parameter is samples-per-channel, not total samples — pass `frames`,
|
||||||
|
// not `frames * channels`), then convert at this boundary.
|
||||||
|
int dec_channels = std::max(1, stream.decoder.channels());
|
||||||
auto maybe_frame = stream.jitter.pop(stream.playout_ts);
|
auto maybe_frame = stream.jitter.pop(stream.playout_ts);
|
||||||
std::vector<int16_t> pcm(frames * params_.channels);
|
std::vector<int16_t> pcm(frames * static_cast<ma_uint32>(dec_channels));
|
||||||
int n;
|
int n;
|
||||||
|
|
||||||
if (maybe_frame) {
|
if (maybe_frame) {
|
||||||
n = stream.decoder.decode(
|
n = stream.decoder.decode(
|
||||||
maybe_frame->payload.data(),
|
maybe_frame->payload.data(),
|
||||||
static_cast<int>(maybe_frame->payload.size()),
|
static_cast<int>(maybe_frame->payload.size()),
|
||||||
pcm.data(), static_cast<int>(pcm.size()));
|
pcm.data(), static_cast<int>(frames));
|
||||||
} else {
|
} else {
|
||||||
n = stream.decoder.decode(nullptr, 0, pcm.data(),
|
n = stream.decoder.decode(nullptr, 0, pcm.data(), static_cast<int>(frames));
|
||||||
static_cast<int>(pcm.size()));
|
|
||||||
}
|
}
|
||||||
|
|
||||||
if (n > 0) {
|
if (n > 0) {
|
||||||
|
// `n` is samples-per-channel (matches the frame_samples convention used by
|
||||||
|
// OpusEncoder::encode elsewhere in the codebase).
|
||||||
|
if (stream.recv_ns)
|
||||||
|
stream.recv_ns->process_capture(pcm.data(), n,
|
||||||
|
static_cast<int>(params_.sample_rate));
|
||||||
|
|
||||||
float g = stream.gain;
|
float g = stream.gain;
|
||||||
for (int i = 0; i < n * static_cast<int>(params_.channels); ++i)
|
for (int i = 0; i < n; ++i) {
|
||||||
mix[i] += static_cast<int32_t>(static_cast<float>(pcm[i]) * g);
|
// Downmix decoder output to the engine's mono accumulator if needed
|
||||||
|
// (average L/R); upmix is unnecessary since the mix buffer is per-channel.
|
||||||
|
int32_t sample = (dec_channels == 2)
|
||||||
|
? (static_cast<int32_t>(pcm[i * 2]) +
|
||||||
|
static_cast<int32_t>(pcm[i * 2 + 1])) / 2
|
||||||
|
: static_cast<int32_t>(pcm[i]);
|
||||||
|
for (uint32_t c = 0; c < params_.channels; ++c)
|
||||||
|
mix[i * params_.channels + c] += static_cast<int32_t>(sample * g);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
stream.playout_ts += frames;
|
stream.playout_ts += frames;
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -31,6 +31,8 @@
|
|||||||
#include "codec/opus_codec.h"
|
#include "codec/opus_codec.h"
|
||||||
#endif
|
#endif
|
||||||
|
|
||||||
|
#include "audio/apm_processor.h"
|
||||||
|
|
||||||
namespace voicecat::audio {
|
namespace voicecat::audio {
|
||||||
|
|
||||||
// ── JitterBuffer ─────────────────────────────────────────────────────────────
|
// ── JitterBuffer ─────────────────────────────────────────────────────────────
|
||||||
@@ -83,8 +85,12 @@ struct AudioParams {
|
|||||||
// Owns miniaudio capture/playback, per-ssrc jitter buffers + Opus decoders, and the mixer.
|
// Owns miniaudio capture/playback, per-ssrc jitter buffers + Opus decoders, and the mixer.
|
||||||
class AudioEngine {
|
class AudioEngine {
|
||||||
public:
|
public:
|
||||||
// Callback type for encoded capture frames ready to be sent.
|
// Callback type for encoded capture frames ready to be sent. `kind` identifies which
|
||||||
using CaptureCallback = std::function<void(const int16_t* pcm, int samples)>;
|
// local stream this PCM belongs to (a vc_stream_kind value; 0 = MIC for the real capture
|
||||||
|
// device, which is always the "primary" tap). M3: multiple concurrent local streams are
|
||||||
|
// possible (e.g. MIC + SCREEN_AUDIO), each fed via its own injection tap (see
|
||||||
|
// inject_capture) since there is only one real hardware capture device.
|
||||||
|
using CaptureCallback = std::function<void(int kind, const int16_t* pcm, int samples)>;
|
||||||
|
|
||||||
AudioEngine();
|
AudioEngine();
|
||||||
~AudioEngine();
|
~AudioEngine();
|
||||||
@@ -100,8 +106,11 @@ class AudioEngine {
|
|||||||
bool running() const { return running_.load(std::memory_order_acquire); }
|
bool running() const { return running_.load(std::memory_order_acquire); }
|
||||||
|
|
||||||
// Inject synthetic PCM directly into the capture pipeline (bypasses real device).
|
// Inject synthetic PCM directly into the capture pipeline (bypasses real device).
|
||||||
// Thread-safe; can be called from any thread including tests.
|
// Thread-safe; can be called from any thread including tests. `kind` selects which local
|
||||||
void inject_capture(const int16_t* pcm, size_t n);
|
// stream's injection tap to feed (each gets its own ring buffer); the 2-arg overload
|
||||||
|
// targets kind 0 (MIC) for source compatibility with existing callers.
|
||||||
|
void inject_capture(int kind, const int16_t* pcm, size_t n);
|
||||||
|
void inject_capture(const int16_t* pcm, size_t n) { inject_capture(0, pcm, n); }
|
||||||
|
|
||||||
// Called by the net thread when a decoded voice frame arrives for a remote stream.
|
// Called by the net thread when a decoded voice frame arrives for a remote stream.
|
||||||
void push_recv_frame(uint32_t ssrc, JitterBuffer::Frame f);
|
void push_recv_frame(uint32_t ssrc, JitterBuffer::Frame f);
|
||||||
@@ -109,8 +118,17 @@ class AudioEngine {
|
|||||||
// Per-stream receive-side controls (safe from any thread).
|
// Per-stream receive-side controls (safe from any thread).
|
||||||
void set_stream_gain(uint32_t ssrc, float gain); // 0.0–2.0, default 1.0
|
void set_stream_gain(uint32_t ssrc, float gain); // 0.0–2.0, default 1.0
|
||||||
void set_stream_mute(uint32_t ssrc, bool mute);
|
void set_stream_mute(uint32_t ssrc, bool mute);
|
||||||
|
// Listener-chosen, local-only noise reduction on a specific remote stream (docs/voice.md
|
||||||
|
// §10) — lazily instantiates an ApmProcessor on first enable, frees it on disable.
|
||||||
|
void set_stream_noise_reduction(uint32_t ssrc, bool enable);
|
||||||
void remove_stream(uint32_t ssrc);
|
void remove_stream(uint32_t ssrc);
|
||||||
|
|
||||||
|
// Edge-triggered talk-state transitions since the last call (docs/voice.md §7: talk state
|
||||||
|
// is derived from recent frame arrival, no protocol message). Call from a lightweight
|
||||||
|
// poller, not the audio callback thread. Returns {ssrc, now_talking} for each stream whose
|
||||||
|
// state flipped.
|
||||||
|
std::vector<std::pair<uint32_t, bool>> poll_talk_transitions();
|
||||||
|
|
||||||
// Get stats for a remote stream's jitter buffer.
|
// Get stats for a remote stream's jitter buffer.
|
||||||
uint32_t stream_packets_lost(uint32_t ssrc) const;
|
uint32_t stream_packets_lost(uint32_t ssrc) const;
|
||||||
uint32_t stream_target_depth_ms(uint32_t ssrc) const;
|
uint32_t stream_target_depth_ms(uint32_t ssrc) const;
|
||||||
@@ -138,12 +156,16 @@ class AudioEngine {
|
|||||||
CaptureCallback capture_cb_;
|
CaptureCallback capture_cb_;
|
||||||
std::atomic<bool> running_{false};
|
std::atomic<bool> running_{false};
|
||||||
|
|
||||||
// Inject ring: stores raw int16 PCM written by inject_capture().
|
// Inject ring(s): stores raw int16 PCM written by inject_capture(), one ring per local
|
||||||
// The encode thread reads from this (no real capture device needed in tests).
|
// stream kind so e.g. MIC and SCREEN_AUDIO can each be fed independently in tests.
|
||||||
|
// The encode thread reads from these (no real capture device needed in tests).
|
||||||
|
struct InjectTap {
|
||||||
|
std::vector<int16_t> ring; // circular, size = kInjectCapSamples
|
||||||
|
std::atomic<size_t> write{0};
|
||||||
|
std::atomic<size_t> read{0};
|
||||||
|
};
|
||||||
std::mutex inject_mu_;
|
std::mutex inject_mu_;
|
||||||
std::vector<int16_t> inject_ring_; // circular, size = frame_samples_
|
std::unordered_map<int, std::unique_ptr<InjectTap>> inject_taps_;
|
||||||
std::atomic<size_t> inject_write_{0};
|
|
||||||
std::atomic<size_t> inject_read_{0};
|
|
||||||
static constexpr size_t kInjectCapSamples = 48000 * 2; // 2 s @48 kHz mono
|
static constexpr size_t kInjectCapSamples = 48000 * 2; // 2 s @48 kHz mono
|
||||||
|
|
||||||
// Per remote stream (protected by streams_mu_).
|
// Per remote stream (protected by streams_mu_).
|
||||||
@@ -155,11 +177,24 @@ class AudioEngine {
|
|||||||
float gain = 1.0f;
|
float gain = 1.0f;
|
||||||
bool mute = false;
|
bool mute = false;
|
||||||
uint32_t playout_ts = 0;
|
uint32_t playout_ts = 0;
|
||||||
|
|
||||||
|
// M3: listener-chosen, local-only noise reduction (docs/voice.md §10). Lazily
|
||||||
|
// created only when enabled — bounded by how many remote streams this listener
|
||||||
|
// subscribes to, so no separate instance cap is needed.
|
||||||
|
bool noise_reduction_enabled = false;
|
||||||
|
std::unique_ptr<ApmProcessor> recv_ns;
|
||||||
|
|
||||||
|
// M3: talk-indicator edge detection (docs/voice.md §7) — updated by push_recv_frame
|
||||||
|
// (already off the real-time audio thread), polled by poll_talk_transitions().
|
||||||
|
std::atomic<int64_t> last_voice_ms{0};
|
||||||
|
bool talking = false;
|
||||||
};
|
};
|
||||||
mutable std::mutex streams_mu_;
|
mutable std::mutex streams_mu_;
|
||||||
std::unordered_map<uint32_t, RemoteStream> streams_;
|
std::unordered_map<uint32_t, RemoteStream> streams_;
|
||||||
|
|
||||||
int frame_samples_ = 960; // 20 ms @48 kHz
|
int frame_samples_ = 960; // 20 ms @48 kHz
|
||||||
|
|
||||||
|
static constexpr int64_t kTalkHangoverMs = 300;
|
||||||
};
|
};
|
||||||
|
|
||||||
} // namespace voicecat::audio
|
} // namespace voicecat::audio
|
||||||
|
|||||||
@@ -11,9 +11,16 @@ bool OpusEncoder::init(const OpusParams& p) {
|
|||||||
channels_ = p.stereo ? 2 : 1;
|
channels_ = p.stereo ? 2 : 1;
|
||||||
frame_samples_ = opus_frame_samples(p);
|
frame_samples_ = opus_frame_samples(p);
|
||||||
|
|
||||||
|
int opus_application = OPUS_APPLICATION_VOIP;
|
||||||
|
switch (p.application) {
|
||||||
|
case OpusApplication::Audio: opus_application = OPUS_APPLICATION_AUDIO; break;
|
||||||
|
case OpusApplication::LowDelay: opus_application = OPUS_APPLICATION_RESTRICTED_LOWDELAY; break;
|
||||||
|
case OpusApplication::Voip: default: opus_application = OPUS_APPLICATION_VOIP; break;
|
||||||
|
}
|
||||||
|
|
||||||
int err = 0;
|
int err = 0;
|
||||||
enc_ = opus_encoder_create(static_cast<opus_int32>(p.sample_rate), channels_,
|
enc_ = opus_encoder_create(static_cast<opus_int32>(p.sample_rate), channels_,
|
||||||
OPUS_APPLICATION_VOIP, &err);
|
opus_application, &err);
|
||||||
if (err != OPUS_OK || !enc_) {
|
if (err != OPUS_OK || !enc_) {
|
||||||
err_ = opus_strerror(err);
|
err_ = opus_strerror(err);
|
||||||
return false;
|
return false;
|
||||||
|
|||||||
@@ -16,6 +16,14 @@
|
|||||||
|
|
||||||
namespace voicecat::codec {
|
namespace voicecat::codec {
|
||||||
|
|
||||||
|
// Mirrors voicecat::v1::OpusApplication (core/proto/voicecat.proto) without depending on
|
||||||
|
// generated protobuf headers from this low-level codec module.
|
||||||
|
enum class OpusApplication {
|
||||||
|
Voip = 0, // speech, optimized for low-rate intelligibility
|
||||||
|
Audio = 1, // music/screen-audio, optimized for fidelity
|
||||||
|
LowDelay = 2, // monitoring, minimal algorithmic delay
|
||||||
|
};
|
||||||
|
|
||||||
struct OpusParams {
|
struct OpusParams {
|
||||||
uint32_t sample_rate = 48000;
|
uint32_t sample_rate = 48000;
|
||||||
uint32_t bitrate_bps = 24000;
|
uint32_t bitrate_bps = 24000;
|
||||||
@@ -25,6 +33,7 @@ struct OpusParams {
|
|||||||
bool dtx = false;
|
bool dtx = false;
|
||||||
uint32_t complexity = 10;
|
uint32_t complexity = 10;
|
||||||
uint32_t expected_packet_loss = 0; // % 0..100
|
uint32_t expected_packet_loss = 0; // % 0..100
|
||||||
|
OpusApplication application = OpusApplication::Voip;
|
||||||
};
|
};
|
||||||
|
|
||||||
// Returns frame_samples for a given sample_rate + frame_ms.
|
// Returns frame_samples for a given sample_rate + frame_ms.
|
||||||
|
|||||||
@@ -241,6 +241,8 @@ cleanup:
|
|||||||
}
|
}
|
||||||
|
|
||||||
void vc_client::teardown_voice() {
|
void vc_client::teardown_voice() {
|
||||||
|
std::lock_guard teardown_lk(teardown_mu_);
|
||||||
|
|
||||||
udp_stop_.store(true, std::memory_order_release);
|
udp_stop_.store(true, std::memory_order_release);
|
||||||
int ufd = udp_fd_.load(std::memory_order_acquire);
|
int ufd = udp_fd_.load(std::memory_order_acquire);
|
||||||
if (ufd != -1) {
|
if (ufd != -1) {
|
||||||
@@ -250,10 +252,21 @@ void vc_client::teardown_voice() {
|
|||||||
if (udp_thread_.joinable()) udp_thread_.join();
|
if (udp_thread_.joinable()) udp_thread_.join();
|
||||||
udp_ready_.store(false, std::memory_order_release);
|
udp_ready_.store(false, std::memory_order_release);
|
||||||
|
|
||||||
|
talk_timer_stop_.store(true, std::memory_order_release);
|
||||||
|
if (talk_timer_thread_.joinable()) talk_timer_thread_.join();
|
||||||
|
|
||||||
audio_engine_.stop();
|
audio_engine_.stop();
|
||||||
local_stream_active_.store(false, std::memory_order_release);
|
{
|
||||||
local_stream_pending_ = false;
|
std::lock_guard lk(local_streams_mu_);
|
||||||
local_encoder_.destroy();
|
for (auto& [kind, ls] : local_streams_) {
|
||||||
|
(void)kind;
|
||||||
|
ls.active.store(false, std::memory_order_release);
|
||||||
|
ls.pending = false;
|
||||||
|
ls.encoder.destroy();
|
||||||
|
}
|
||||||
|
local_streams_.clear();
|
||||||
|
pending_announce_kind_.clear();
|
||||||
|
}
|
||||||
|
|
||||||
media_send_crypto_.reset();
|
media_send_crypto_.reset();
|
||||||
media_recv_crypto_.reset();
|
media_recv_crypto_.reset();
|
||||||
@@ -314,7 +327,7 @@ void vc_client::handle_envelope(const voicecat::v1::Envelope& env) {
|
|||||||
handle_udp_binding_ack(env.udp_binding());
|
handle_udp_binding_ack(env.udp_binding());
|
||||||
break;
|
break;
|
||||||
case voicecat::v1::Envelope::kStreamAnnounceResult:
|
case voicecat::v1::Envelope::kStreamAnnounceResult:
|
||||||
handle_stream_announce_result(env.stream_announce_result());
|
handle_stream_announce_result(env.request_id(), env.stream_announce_result());
|
||||||
break;
|
break;
|
||||||
case voicecat::v1::Envelope::kPong:
|
case voicecat::v1::Envelope::kPong:
|
||||||
break; // ignore keepalive responses
|
break; // ignore keepalive responses
|
||||||
@@ -565,6 +578,9 @@ void vc_client::finish_udp_binding() {
|
|||||||
udp_stop_.store(false, std::memory_order_release);
|
udp_stop_.store(false, std::memory_order_release);
|
||||||
udp_ready_.store(true, std::memory_order_release);
|
udp_ready_.store(true, std::memory_order_release);
|
||||||
udp_thread_ = std::thread([this] { run_udp_recv(); });
|
udp_thread_ = std::thread([this] { run_udp_recv(); });
|
||||||
|
|
||||||
|
talk_timer_stop_.store(false, std::memory_order_release);
|
||||||
|
talk_timer_thread_ = std::thread([this] { run_talk_timer(); });
|
||||||
}
|
}
|
||||||
|
|
||||||
void vc_client::run_udp_recv() {
|
void vc_client::run_udp_recv() {
|
||||||
@@ -608,22 +624,67 @@ void vc_client::run_udp_recv() {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
void vc_client::on_capture_frame(const int16_t* pcm, int samples) {
|
namespace {
|
||||||
if (!local_stream_active_.load(std::memory_order_acquire)) return;
|
int64_t client_now_ms() {
|
||||||
if (self_mic_muted_.load(std::memory_order_acquire)) return;
|
return std::chrono::duration_cast<std::chrono::milliseconds>(
|
||||||
|
std::chrono::steady_clock::now().time_since_epoch())
|
||||||
|
.count();
|
||||||
|
}
|
||||||
|
|
||||||
|
// Builds an OpusParams from a wire AudioConfig, applying the same field-by-field mapping on
|
||||||
|
// both the send (local-stream encoder) and receive (remote-stream decoder) paths — fixes the
|
||||||
|
// M2 gap where mode/dtx/complexity/application were silently dropped.
|
||||||
|
voicecat::codec::OpusParams opus_params_from_audio_config(const voicecat::v1::AudioConfig& a) {
|
||||||
|
voicecat::codec::OpusParams p;
|
||||||
|
p.sample_rate = a.sample_rate() ? a.sample_rate() : 48000;
|
||||||
|
p.bitrate_bps = a.bitrate_bps() ? a.bitrate_bps() : 24000;
|
||||||
|
p.frame_ms = a.frame_ms() ? a.frame_ms() : 20;
|
||||||
|
p.stereo = (a.mode() == voicecat::v1::MODE_STEREO);
|
||||||
|
p.fec = a.fec();
|
||||||
|
p.dtx = a.dtx();
|
||||||
|
p.complexity = a.complexity() ? a.complexity() : 10;
|
||||||
|
p.expected_packet_loss = a.expected_packet_loss();
|
||||||
|
p.application = static_cast<voicecat::codec::OpusApplication>(a.application());
|
||||||
|
return p;
|
||||||
|
}
|
||||||
|
} // namespace
|
||||||
|
|
||||||
|
void vc_client::on_capture_frame(int kind, const int16_t* pcm, int samples) {
|
||||||
|
std::lock_guard lk(local_streams_mu_);
|
||||||
|
auto it = local_streams_.find(kind);
|
||||||
|
if (it == local_streams_.end() || !it->second.active.load(std::memory_order_acquire)) return;
|
||||||
|
// "Mic muted" only gates the MIC stream — a concurrently-running SCREEN_AUDIO share keeps
|
||||||
|
// playing while the user's mic is muted (docs §M3 scope decision).
|
||||||
|
if (kind == static_cast<int>(VC_STREAM_MIC) &&
|
||||||
|
self_mic_muted_.load(std::memory_order_acquire)) return;
|
||||||
if (!media_send_crypto_) return;
|
if (!media_send_crypto_) return;
|
||||||
int fd = udp_fd_.load(std::memory_order_acquire);
|
int fd = udp_fd_.load(std::memory_order_acquire);
|
||||||
if (fd == -1) return;
|
if (fd == -1) return;
|
||||||
|
|
||||||
|
auto& ls = it->second;
|
||||||
|
ls.last_capture_ms.store(client_now_ms(), std::memory_order_relaxed);
|
||||||
|
|
||||||
uint8_t opus_buf[1500];
|
uint8_t opus_buf[1500];
|
||||||
int opus_len = local_encoder_.encode(pcm, samples, opus_buf, sizeof(opus_buf));
|
int opus_len;
|
||||||
|
if (ls.effective_params.stereo) {
|
||||||
|
// Capture is always mono in M3 (no stereo capture device); upmix L=R so a
|
||||||
|
// channel configured for stereo still gets a real, spec-correct stereo Opus stream.
|
||||||
|
std::vector<int16_t> stereo_pcm(static_cast<size_t>(samples) * 2);
|
||||||
|
for (int i = 0; i < samples; ++i) {
|
||||||
|
stereo_pcm[i * 2] = pcm[i];
|
||||||
|
stereo_pcm[i * 2 + 1] = pcm[i];
|
||||||
|
}
|
||||||
|
opus_len = ls.encoder.encode(stereo_pcm.data(), samples, opus_buf, sizeof(opus_buf));
|
||||||
|
} else {
|
||||||
|
opus_len = ls.encoder.encode(pcm, samples, opus_buf, sizeof(opus_buf));
|
||||||
|
}
|
||||||
if (opus_len <= 0) return;
|
if (opus_len <= 0) return;
|
||||||
|
|
||||||
voicecat::net::VoiceFrame hdr;
|
voicecat::net::VoiceFrame hdr;
|
||||||
hdr.ssrc = local_ssrc_;
|
hdr.ssrc = ls.ssrc;
|
||||||
hdr.seq = static_cast<uint16_t>(media_send_crypto_->peek_send_counter());
|
hdr.seq = static_cast<uint16_t>(media_send_crypto_->peek_send_counter());
|
||||||
hdr.timestamp = local_timestamp_;
|
hdr.timestamp = ls.timestamp;
|
||||||
local_timestamp_ += static_cast<uint32_t>(samples);
|
ls.timestamp += static_cast<uint32_t>(samples);
|
||||||
|
|
||||||
uint8_t header_bytes[voicecat::net::kVoiceHeaderSize];
|
uint8_t header_bytes[voicecat::net::kVoiceHeaderSize];
|
||||||
voicecat::net::serialize_header(hdr, header_bytes);
|
voicecat::net::serialize_header(hdr, header_bytes);
|
||||||
@@ -653,7 +714,9 @@ void vc_client::ensure_audio_running() {
|
|||||||
p.sample_rate = 48000;
|
p.sample_rate = 48000;
|
||||||
p.channels = 1;
|
p.channels = 1;
|
||||||
p.frame_ms = 20;
|
p.frame_ms = 20;
|
||||||
audio_engine_.start(p, [this](const int16_t* pcm, int samples) { on_capture_frame(pcm, samples); });
|
audio_engine_.start(p, [this](int kind, const int16_t* pcm, int samples) {
|
||||||
|
on_capture_frame(kind, pcm, samples);
|
||||||
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
void vc_client::sync_remote_streams(const voicecat::v1::User& user) {
|
void vc_client::sync_remote_streams(const voicecat::v1::User& user) {
|
||||||
@@ -671,9 +734,7 @@ void vc_client::sync_remote_streams(const voicecat::v1::User& user) {
|
|||||||
|
|
||||||
remote_streams_[ssrc] = {user.id(), si.stream_id()};
|
remote_streams_[ssrc] = {user.id(), si.stream_id()};
|
||||||
|
|
||||||
voicecat::codec::OpusParams p;
|
voicecat::codec::OpusParams p = opus_params_from_audio_config(si.audio());
|
||||||
p.sample_rate = si.audio().sample_rate() ? si.audio().sample_rate() : 48000;
|
|
||||||
p.frame_ms = si.audio().frame_ms() ? si.audio().frame_ms() : 20;
|
|
||||||
audio_engine_.init_recv_stream(ssrc, p);
|
audio_engine_.init_recv_stream(ssrc, p);
|
||||||
audio_engine_.set_stream_mute(ssrc, self_deafened_.load(std::memory_order_acquire));
|
audio_engine_.set_stream_mute(ssrc, self_deafened_.load(std::memory_order_acquire));
|
||||||
|
|
||||||
@@ -714,16 +775,26 @@ void vc_client::sync_remote_streams(const voicecat::v1::User& user) {
|
|||||||
|
|
||||||
vc_result vc_client::stream_start(const vc_stream_desc& desc, uint32_t* out_stream_id) {
|
vc_result vc_client::stream_start(const vc_stream_desc& desc, uint32_t* out_stream_id) {
|
||||||
if (state_net_.load(std::memory_order_acquire) != VC_STATE_CONNECTED) return VC_ERR_NOT_CONNECTED;
|
if (state_net_.load(std::memory_order_acquire) != VC_STATE_CONNECTED) return VC_ERR_NOT_CONNECTED;
|
||||||
if (local_stream_active_.load(std::memory_order_acquire) || local_stream_pending_)
|
|
||||||
return VC_ERR_ALREADY;
|
int kind = static_cast<int>(desc.kind);
|
||||||
|
uint64_t req_id;
|
||||||
|
{
|
||||||
|
std::lock_guard lk(local_streams_mu_);
|
||||||
|
auto& ls = local_streams_[kind]; // default-constructs if this kind is new
|
||||||
|
if (ls.active.load(std::memory_order_acquire) || ls.pending) return VC_ERR_ALREADY;
|
||||||
|
|
||||||
uint32_t sid = next_local_stream_id_++;
|
uint32_t sid = next_local_stream_id_++;
|
||||||
local_stream_id_ = sid;
|
ls.stream_id = sid;
|
||||||
local_stream_pending_ = true;
|
ls.pending = true;
|
||||||
if (out_stream_id) *out_stream_id = sid;
|
if (out_stream_id) *out_stream_id = sid;
|
||||||
|
|
||||||
|
req_id = next_req_id_++;
|
||||||
|
ls.pending_request_id = req_id;
|
||||||
|
pending_announce_kind_[req_id] = kind;
|
||||||
|
}
|
||||||
|
|
||||||
voicecat::v1::Envelope req;
|
voicecat::v1::Envelope req;
|
||||||
req.set_request_id(next_req_id_++);
|
req.set_request_id(req_id);
|
||||||
auto* ann = req.mutable_stream_announce();
|
auto* ann = req.mutable_stream_announce();
|
||||||
ann->set_kind(desc.kind == VC_STREAM_SCREEN_AUDIO ? voicecat::v1::STREAM_SCREEN_AUDIO
|
ann->set_kind(desc.kind == VC_STREAM_SCREEN_AUDIO ? voicecat::v1::STREAM_SCREEN_AUDIO
|
||||||
: desc.kind == VC_STREAM_AUX_DEVICE ? voicecat::v1::STREAM_AUX_DEVICE
|
: desc.kind == VC_STREAM_AUX_DEVICE ? voicecat::v1::STREAM_AUX_DEVICE
|
||||||
@@ -731,55 +802,76 @@ vc_result vc_client::stream_start(const vc_stream_desc& desc, uint32_t* out_stre
|
|||||||
if (desc.label) ann->set_label(desc.label);
|
if (desc.label) ann->set_label(desc.label);
|
||||||
auto* audio = ann->mutable_requested_audio();
|
auto* audio = ann->mutable_requested_audio();
|
||||||
audio->set_sample_rate(48000);
|
audio->set_sample_rate(48000);
|
||||||
audio->set_bitrate_bps(24000);
|
// bitrate_bps intentionally left unset (0 = "no preference"): the channel's AudioConfig
|
||||||
|
// is authoritative when the channel has one (docs/voice.md §3) — a client-side default
|
||||||
|
// here would otherwise always clamp the channel's bitrate down to this value. frame_ms/
|
||||||
|
// fec are likewise channel-controlled when a channel config exists; these are only used
|
||||||
|
// as a fallback when it doesn't.
|
||||||
audio->set_frame_ms(20);
|
audio->set_frame_ms(20);
|
||||||
audio->set_fec(true);
|
audio->set_fec(true);
|
||||||
queue_envelope(req);
|
queue_envelope(req);
|
||||||
return VC_OK;
|
return VC_OK;
|
||||||
}
|
}
|
||||||
|
|
||||||
void vc_client::handle_stream_announce_result(const voicecat::v1::StreamAnnounceResult& msg) {
|
void vc_client::handle_stream_announce_result(uint64_t req_id,
|
||||||
if (!local_stream_pending_) return;
|
const voicecat::v1::StreamAnnounceResult& msg) {
|
||||||
local_stream_pending_ = false;
|
uint32_t self_uid;
|
||||||
|
uint32_t emit_stream_id;
|
||||||
|
bool ok_to_emit = false;
|
||||||
|
{
|
||||||
|
std::lock_guard lk(local_streams_mu_);
|
||||||
|
auto pit = pending_announce_kind_.find(req_id);
|
||||||
|
if (pit == pending_announce_kind_.end()) return; // stray/duplicate — ignore
|
||||||
|
int kind = pit->second;
|
||||||
|
pending_announce_kind_.erase(pit);
|
||||||
|
|
||||||
|
auto sit = local_streams_.find(kind);
|
||||||
|
if (sit == local_streams_.end() || !sit->second.pending) return;
|
||||||
|
auto& ls = sit->second;
|
||||||
|
ls.pending = false;
|
||||||
|
|
||||||
if (!msg.ok()) {
|
if (!msg.ok()) {
|
||||||
emit_error(VC_ERR_PROTOCOL, msg.error().c_str());
|
emit_error(VC_ERR_PROTOCOL, msg.error().c_str());
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
local_ssrc_ = msg.ssrc();
|
ls.ssrc = msg.ssrc();
|
||||||
local_timestamp_ = 0;
|
ls.timestamp = 0;
|
||||||
|
ls.effective_params = opus_params_from_audio_config(msg.effective_audio());
|
||||||
|
ls.frame_samples = static_cast<uint16_t>(voicecat::codec::opus_frame_samples(ls.effective_params));
|
||||||
|
|
||||||
voicecat::codec::OpusParams p;
|
if (!ls.encoder.init(ls.effective_params)) {
|
||||||
const auto& eff = msg.effective_audio();
|
|
||||||
p.sample_rate = eff.sample_rate() ? eff.sample_rate() : 48000;
|
|
||||||
p.bitrate_bps = eff.bitrate_bps() ? eff.bitrate_bps() : 24000;
|
|
||||||
p.frame_ms = eff.frame_ms() ? eff.frame_ms() : 20;
|
|
||||||
p.fec = eff.fec();
|
|
||||||
local_frame_samples_ = static_cast<uint16_t>(voicecat::codec::opus_frame_samples(p));
|
|
||||||
|
|
||||||
if (!local_encoder_.init(p)) {
|
|
||||||
emit_error(VC_ERR_AUDIO, "opus encoder init failed");
|
emit_error(VC_ERR_AUDIO, "opus encoder init failed");
|
||||||
return;
|
return;
|
||||||
}
|
}
|
||||||
|
|
||||||
ensure_audio_running();
|
ls.active.store(true, std::memory_order_release);
|
||||||
local_stream_active_.store(true, std::memory_order_release);
|
self_uid = self_user_id_;
|
||||||
|
emit_stream_id = ls.stream_id;
|
||||||
|
ok_to_emit = true;
|
||||||
|
}
|
||||||
|
|
||||||
|
ensure_audio_running();
|
||||||
|
|
||||||
|
if (ok_to_emit) {
|
||||||
vc_event ev{};
|
vc_event ev{};
|
||||||
ev.type = VC_EVENT_STREAM_STARTED;
|
ev.type = VC_EVENT_STREAM_STARTED;
|
||||||
ev.user_id = self_user_id_;
|
ev.user_id = self_uid;
|
||||||
ev.stream_id = local_stream_id_;
|
ev.stream_id = emit_stream_id;
|
||||||
emit(ev);
|
emit(ev);
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
vc_result vc_client::stream_stop(uint32_t stream_id) {
|
vc_result vc_client::stream_stop(uint32_t stream_id) {
|
||||||
if (state_net_.load(std::memory_order_acquire) != VC_STATE_CONNECTED) return VC_ERR_NOT_CONNECTED;
|
if (state_net_.load(std::memory_order_acquire) != VC_STATE_CONNECTED) return VC_ERR_NOT_CONNECTED;
|
||||||
if (!local_stream_active_.load(std::memory_order_acquire) || stream_id != local_stream_id_)
|
|
||||||
return VC_ERR_INVALID_ARG;
|
|
||||||
|
|
||||||
local_stream_active_.store(false, std::memory_order_release);
|
{
|
||||||
local_encoder_.destroy();
|
std::lock_guard lk(local_streams_mu_);
|
||||||
|
LocalStream* ls = find_local_stream_by_id(stream_id);
|
||||||
|
if (!ls || !ls->active.load(std::memory_order_acquire)) return VC_ERR_INVALID_ARG;
|
||||||
|
ls->active.store(false, std::memory_order_release);
|
||||||
|
ls->encoder.destroy();
|
||||||
|
}
|
||||||
|
|
||||||
voicecat::v1::Envelope req;
|
voicecat::v1::Envelope req;
|
||||||
req.set_request_id(next_req_id_++);
|
req.set_request_id(next_req_id_++);
|
||||||
@@ -794,6 +886,14 @@ vc_result vc_client::stream_stop(uint32_t stream_id) {
|
|||||||
return VC_OK;
|
return VC_OK;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
vc_client::LocalStream* vc_client::find_local_stream_by_id(uint32_t stream_id) {
|
||||||
|
for (auto& [kind, ls] : local_streams_) {
|
||||||
|
(void)kind;
|
||||||
|
if (ls.stream_id == stream_id) return &ls;
|
||||||
|
}
|
||||||
|
return nullptr;
|
||||||
|
}
|
||||||
|
|
||||||
vc_result vc_client::set_input_device(uint32_t, const char*) { return VC_ERR_NOT_IMPLEMENTED; }
|
vc_result vc_client::set_input_device(uint32_t, const char*) { return VC_ERR_NOT_IMPLEMENTED; }
|
||||||
vc_result vc_client::set_input_mode(vc_input_mode) { return VC_ERR_NOT_IMPLEMENTED; }
|
vc_result vc_client::set_input_mode(vc_input_mode) { return VC_ERR_NOT_IMPLEMENTED; }
|
||||||
vc_result vc_client::set_push_to_talk(bool) { return VC_ERR_NOT_IMPLEMENTED; }
|
vc_result vc_client::set_push_to_talk(bool) { return VC_ERR_NOT_IMPLEMENTED; }
|
||||||
@@ -812,7 +912,7 @@ vc_result vc_client::set_self_mute(bool mic_muted, bool deafened) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
vc_result vc_client::set_remote_stream(uint32_t user_id, uint32_t stream_id, float gain,
|
vc_result vc_client::set_remote_stream(uint32_t user_id, uint32_t stream_id, float gain,
|
||||||
bool muted, bool /*noise_reduction*/) {
|
bool muted, bool noise_reduction) {
|
||||||
if (state_net_.load(std::memory_order_acquire) != VC_STATE_CONNECTED) return VC_ERR_NOT_CONNECTED;
|
if (state_net_.load(std::memory_order_acquire) != VC_STATE_CONNECTED) return VC_ERR_NOT_CONNECTED;
|
||||||
const auto* user = session_model_.find_user(user_id);
|
const auto* user = session_model_.find_user(user_id);
|
||||||
if (!user) return VC_ERR_INVALID_ARG;
|
if (!user) return VC_ERR_INVALID_ARG;
|
||||||
@@ -820,18 +920,124 @@ vc_result vc_client::set_remote_stream(uint32_t user_id, uint32_t stream_id, flo
|
|||||||
if (s.stream_id == stream_id) {
|
if (s.stream_id == stream_id) {
|
||||||
audio_engine_.set_stream_gain(s.ssrc, gain);
|
audio_engine_.set_stream_gain(s.ssrc, gain);
|
||||||
audio_engine_.set_stream_mute(s.ssrc, muted);
|
audio_engine_.set_stream_mute(s.ssrc, muted);
|
||||||
|
audio_engine_.set_stream_noise_reduction(s.ssrc, noise_reduction);
|
||||||
return VC_OK;
|
return VC_OK;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
return VC_ERR_INVALID_ARG;
|
return VC_ERR_INVALID_ARG;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
vc_result vc_client::get_stream_audio_config(uint32_t user_id, uint32_t stream_id,
|
||||||
|
vc_audio_config* out) {
|
||||||
|
if (state_net_.load(std::memory_order_acquire) != VC_STATE_CONNECTED) return VC_ERR_NOT_CONNECTED;
|
||||||
|
|
||||||
|
if (user_id == self_user_id_) {
|
||||||
|
std::lock_guard lk(local_streams_mu_);
|
||||||
|
LocalStream* ls = find_local_stream_by_id(stream_id);
|
||||||
|
if (!ls || !ls->active.load(std::memory_order_acquire)) return VC_ERR_INVALID_ARG;
|
||||||
|
const auto& p = ls->effective_params;
|
||||||
|
out->codec = 0;
|
||||||
|
out->mode = p.stereo ? 1u : 0u;
|
||||||
|
out->sample_rate = p.sample_rate;
|
||||||
|
out->bitrate_bps = p.bitrate_bps;
|
||||||
|
out->frame_ms = p.frame_ms;
|
||||||
|
out->application = static_cast<uint32_t>(p.application);
|
||||||
|
out->fec = p.fec ? 1 : 0;
|
||||||
|
out->expected_packet_loss = p.expected_packet_loss;
|
||||||
|
out->dtx = p.dtx ? 1 : 0;
|
||||||
|
out->complexity = p.complexity;
|
||||||
|
return VC_OK;
|
||||||
|
}
|
||||||
|
|
||||||
|
const auto* user = session_model_.find_user(user_id);
|
||||||
|
if (!user) return VC_ERR_INVALID_ARG;
|
||||||
|
for (const auto& s : user->streams) {
|
||||||
|
if (s.stream_id != stream_id) continue;
|
||||||
|
out->codec = 0;
|
||||||
|
out->mode = s.mode;
|
||||||
|
out->sample_rate = s.sample_rate;
|
||||||
|
out->bitrate_bps = s.bitrate_bps;
|
||||||
|
out->frame_ms = s.frame_ms;
|
||||||
|
out->application = s.application;
|
||||||
|
out->fec = s.fec ? 1 : 0;
|
||||||
|
out->expected_packet_loss = s.expected_packet_loss;
|
||||||
|
out->dtx = s.dtx ? 1 : 0;
|
||||||
|
out->complexity = s.complexity;
|
||||||
|
return VC_OK;
|
||||||
|
}
|
||||||
|
return VC_ERR_INVALID_ARG;
|
||||||
|
}
|
||||||
|
|
||||||
|
vc_result vc_client::test_inject_capture(uint32_t stream_id, const int16_t* pcm, size_t samples) {
|
||||||
|
int kind = -1;
|
||||||
|
{
|
||||||
|
std::lock_guard lk(local_streams_mu_);
|
||||||
|
for (auto& [k, ls] : local_streams_) {
|
||||||
|
if (ls.stream_id == stream_id && ls.active.load(std::memory_order_acquire)) {
|
||||||
|
kind = k;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if (kind < 0) return VC_ERR_INVALID_ARG;
|
||||||
|
audio_engine_.inject_capture(kind, pcm, samples);
|
||||||
|
return VC_OK;
|
||||||
|
}
|
||||||
|
|
||||||
vc_result vc_client::list_devices(vc_device_kind, vc_device_list* out) {
|
vc_result vc_client::list_devices(vc_device_kind, vc_device_list* out) {
|
||||||
out->items = nullptr;
|
out->items = nullptr;
|
||||||
out->count = 0;
|
out->count = 0;
|
||||||
return VC_ERR_NOT_IMPLEMENTED;
|
return VC_ERR_NOT_IMPLEMENTED;
|
||||||
}
|
}
|
||||||
|
|
||||||
|
void vc_client::run_talk_timer() {
|
||||||
|
while (!talk_timer_stop_.load(std::memory_order_acquire)) {
|
||||||
|
// Remote streams: ask the engine for edge-triggered transitions, map ssrc -> (user,
|
||||||
|
// stream) via remote_streams_, emit.
|
||||||
|
for (auto& [ssrc, talking] : audio_engine_.poll_talk_transitions()) {
|
||||||
|
uint32_t uid = 0, sid = 0;
|
||||||
|
{
|
||||||
|
std::lock_guard lk(remote_streams_mu_);
|
||||||
|
auto it = remote_streams_.find(ssrc);
|
||||||
|
if (it == remote_streams_.end()) continue;
|
||||||
|
uid = it->second.first;
|
||||||
|
sid = it->second.second;
|
||||||
|
}
|
||||||
|
vc_event ev{};
|
||||||
|
ev.type = VC_EVENT_TALK_STATE;
|
||||||
|
ev.user_id = uid;
|
||||||
|
ev.stream_id = sid;
|
||||||
|
ev.u32a = talking ? 1u : 0u;
|
||||||
|
emit(ev);
|
||||||
|
}
|
||||||
|
|
||||||
|
// Local streams: same hangover logic, driven by on_capture_frame's last_capture_ms.
|
||||||
|
std::vector<vc_event> local_events;
|
||||||
|
{
|
||||||
|
std::lock_guard lk(local_streams_mu_);
|
||||||
|
int64_t now = client_now_ms();
|
||||||
|
for (auto& [kind, ls] : local_streams_) {
|
||||||
|
(void)kind;
|
||||||
|
if (!ls.active.load(std::memory_order_acquire)) continue;
|
||||||
|
bool now_talking =
|
||||||
|
(now - ls.last_capture_ms.load(std::memory_order_relaxed)) < kTalkHangoverMs;
|
||||||
|
if (now_talking != ls.talking) {
|
||||||
|
ls.talking = now_talking;
|
||||||
|
vc_event ev{};
|
||||||
|
ev.type = VC_EVENT_TALK_STATE;
|
||||||
|
ev.user_id = self_user_id_;
|
||||||
|
ev.stream_id = ls.stream_id;
|
||||||
|
ev.u32a = now_talking ? 1u : 0u;
|
||||||
|
local_events.push_back(ev);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
for (auto& ev : local_events) emit(ev);
|
||||||
|
|
||||||
|
std::this_thread::sleep_for(std::chrono::milliseconds(kTalkPollMs));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
#else // !VOICECAT_HAS_NET
|
#else // !VOICECAT_HAS_NET
|
||||||
|
|
||||||
// ── M0 stub implementations ───────────────────────────────────────────────────
|
// ── M0 stub implementations ───────────────────────────────────────────────────
|
||||||
@@ -864,5 +1070,11 @@ vc_result vc_client::list_devices(vc_device_kind, vc_device_list* out) {
|
|||||||
out->count = 0;
|
out->count = 0;
|
||||||
return VC_ERR_NOT_IMPLEMENTED;
|
return VC_ERR_NOT_IMPLEMENTED;
|
||||||
}
|
}
|
||||||
|
vc_result vc_client::get_stream_audio_config(uint32_t, uint32_t, vc_audio_config*) {
|
||||||
|
return VC_ERR_NOT_IMPLEMENTED;
|
||||||
|
}
|
||||||
|
vc_result vc_client::test_inject_capture(uint32_t, const int16_t*, size_t) {
|
||||||
|
return VC_ERR_NOT_IMPLEMENTED;
|
||||||
|
}
|
||||||
|
|
||||||
#endif // VOICECAT_HAS_NET
|
#endif // VOICECAT_HAS_NET
|
||||||
|
|||||||
@@ -58,6 +58,14 @@ struct vc_client {
|
|||||||
|
|
||||||
vc_result list_devices(vc_device_kind kind, vc_device_list* out);
|
vc_result list_devices(vc_device_kind kind, vc_device_list* out);
|
||||||
|
|
||||||
|
// M3: effective Opus config for a (user_id, stream_id) — our own pending/active local
|
||||||
|
// streams, or any peer's broadcast StreamInfo.audio.
|
||||||
|
vc_result get_stream_audio_config(uint32_t user_id, uint32_t stream_id,
|
||||||
|
vc_audio_config* out);
|
||||||
|
|
||||||
|
// TEST-ONLY (see voicecat.h) — inject synthetic PCM into a local stream's encode pipeline.
|
||||||
|
vc_result test_inject_capture(uint32_t stream_id, const int16_t* pcm, size_t samples);
|
||||||
|
|
||||||
vc_connection_state state() const {
|
vc_connection_state state() const {
|
||||||
#ifdef VOICECAT_HAS_NET
|
#ifdef VOICECAT_HAS_NET
|
||||||
return state_net_.load(std::memory_order_acquire);
|
return state_net_.load(std::memory_order_acquire);
|
||||||
@@ -122,20 +130,43 @@ struct vc_client {
|
|||||||
std::unique_ptr<voicecat::crypto::SodiumMediaCrypto> media_recv_crypto_;
|
std::unique_ptr<voicecat::crypto::SodiumMediaCrypto> media_recv_crypto_;
|
||||||
|
|
||||||
voicecat::audio::AudioEngine audio_engine_;
|
voicecat::audio::AudioEngine audio_engine_;
|
||||||
voicecat::codec::OpusEncoder local_encoder_;
|
|
||||||
|
|
||||||
std::atomic<bool> local_stream_active_{false};
|
// M3: one LocalStream per concurrently-active stream kind (MIC/SCREEN_AUDIO/AUX_DEVICE
|
||||||
bool local_stream_pending_{false}; // announced, waiting for result
|
// are each singletons for a given client — see docs/roadmap.md §M3). Replaces the M2
|
||||||
uint32_t local_stream_id_{0};
|
// single-stream fields (local_encoder_/local_stream_active_/etc).
|
||||||
uint32_t local_ssrc_{0};
|
struct LocalStream {
|
||||||
|
voicecat::codec::OpusEncoder encoder;
|
||||||
|
std::atomic<bool> active{false};
|
||||||
|
bool pending{false}; // announced, awaiting StreamAnnounceResult
|
||||||
|
uint32_t stream_id{0};
|
||||||
|
uint32_t ssrc{0};
|
||||||
|
uint32_t timestamp{0};
|
||||||
|
uint16_t frame_samples{960};
|
||||||
|
uint64_t pending_request_id{0};
|
||||||
|
// Full effective OpusParams from the last StreamAnnounceResult — retained so
|
||||||
|
// vc_get_stream_audio_config() has something to read back for our own streams.
|
||||||
|
voicecat::codec::OpusParams effective_params;
|
||||||
|
|
||||||
|
// Talk-indicator edge detection (docs/voice.md §7) — updated in on_capture_frame.
|
||||||
|
std::atomic<int64_t> last_capture_ms{0};
|
||||||
|
bool talking = false;
|
||||||
|
};
|
||||||
|
mutable std::mutex local_streams_mu_;
|
||||||
|
std::unordered_map<int, LocalStream> local_streams_; // keyed by vc_stream_kind
|
||||||
|
std::unordered_map<uint64_t, int> pending_announce_kind_; // request_id -> kind
|
||||||
uint32_t next_local_stream_id_{1};
|
uint32_t next_local_stream_id_{1};
|
||||||
uint32_t local_timestamp_{0};
|
|
||||||
uint16_t local_frame_samples_{960};
|
|
||||||
|
|
||||||
// ssrc → (user_id, stream_id) for remote streams already wired into audio_engine_.
|
// ssrc → (user_id, stream_id) for remote streams already wired into audio_engine_.
|
||||||
mutable std::mutex remote_streams_mu_;
|
mutable std::mutex remote_streams_mu_;
|
||||||
std::unordered_map<uint32_t, std::pair<uint32_t, uint32_t>> remote_streams_;
|
std::unordered_map<uint32_t, std::pair<uint32_t, uint32_t>> remote_streams_;
|
||||||
|
|
||||||
|
// M3: talk-indicator polling thread (separate from udp_thread_ / the miniaudio callback
|
||||||
|
// thread — see architecture.md §3 real-time rule).
|
||||||
|
std::thread talk_timer_thread_;
|
||||||
|
std::atomic<bool> talk_timer_stop_{false};
|
||||||
|
static constexpr int64_t kTalkPollMs = 100;
|
||||||
|
static constexpr int64_t kTalkHangoverMs = 300;
|
||||||
|
|
||||||
// Local UDP destination (server media endpoint), resolved once during binding.
|
// Local UDP destination (server media endpoint), resolved once during binding.
|
||||||
uint32_t udp_dest_addr_{0}; // network byte order
|
uint32_t udp_dest_addr_{0}; // network byte order
|
||||||
uint16_t udp_dest_port_{0}; // network byte order
|
uint16_t udp_dest_port_{0}; // network byte order
|
||||||
@@ -143,6 +174,14 @@ struct vc_client {
|
|||||||
std::atomic<bool> self_mic_muted_{false};
|
std::atomic<bool> self_mic_muted_{false};
|
||||||
std::atomic<bool> self_deafened_{false};
|
std::atomic<bool> self_deafened_{false};
|
||||||
|
|
||||||
|
// teardown_voice() is called both from run_io()'s own cleanup (on the io_thread_, when
|
||||||
|
// the read loop exits) and from disconnect() (on the caller's thread) -- without
|
||||||
|
// serializing those two call sites, both can see udp_thread_/talk_timer_thread_ as
|
||||||
|
// joinable() at the same time and race to join() the same std::thread object (UB; an
|
||||||
|
// intermittent "No such process" std::system_error on Windows is the typical symptom).
|
||||||
|
// This mutex makes teardown_voice() idempotent under concurrent calls.
|
||||||
|
std::mutex teardown_mu_;
|
||||||
|
|
||||||
// ── io_thread_ entry point ──────────────────────────────────────────────────
|
// ── io_thread_ entry point ──────────────────────────────────────────────────
|
||||||
void run_io(std::string host, uint16_t port);
|
void run_io(std::string host, uint16_t port);
|
||||||
|
|
||||||
@@ -155,7 +194,8 @@ struct vc_client {
|
|||||||
void handle_text_message(const voicecat::v1::TextMessage& msg);
|
void handle_text_message(const voicecat::v1::TextMessage& msg);
|
||||||
void handle_disconnect(const voicecat::v1::Disconnect& msg);
|
void handle_disconnect(const voicecat::v1::Disconnect& msg);
|
||||||
void handle_udp_binding_ack(const voicecat::v1::UdpBinding& msg);
|
void handle_udp_binding_ack(const voicecat::v1::UdpBinding& msg);
|
||||||
void handle_stream_announce_result(const voicecat::v1::StreamAnnounceResult& msg);
|
void handle_stream_announce_result(uint64_t req_id,
|
||||||
|
const voicecat::v1::StreamAnnounceResult& msg);
|
||||||
|
|
||||||
// ── M2: UDP / media helpers ──────────────────────────────────────────────────
|
// ── M2: UDP / media helpers ──────────────────────────────────────────────────
|
||||||
// Kicks off TCP UdpBinding request; called once after a successful AuthResult.
|
// Kicks off TCP UdpBinding request; called once after a successful AuthResult.
|
||||||
@@ -164,8 +204,9 @@ struct vc_client {
|
|||||||
void finish_udp_binding();
|
void finish_udp_binding();
|
||||||
// udp_thread_ entry point: recv loop, AEAD-open, decode, push to audio_engine_.
|
// udp_thread_ entry point: recv loop, AEAD-open, decode, push to audio_engine_.
|
||||||
void run_udp_recv();
|
void run_udp_recv();
|
||||||
// capture_cb passed to audio_engine_.start(): encode + seal + send one frame.
|
// capture_cb passed to audio_engine_.start(): encode + seal + send one frame for the
|
||||||
void on_capture_frame(const int16_t* pcm, int samples);
|
// given local stream `kind` (M3: multiple concurrent local streams are possible).
|
||||||
|
void on_capture_frame(int kind, const int16_t* pcm, int samples);
|
||||||
// Inspect a User proto's streams and wire up any new remote ssrc into audio_engine_,
|
// Inspect a User proto's streams and wire up any new remote ssrc into audio_engine_,
|
||||||
// emitting VC_EVENT_STREAM_STARTED/STOPPED as streams appear/disappear.
|
// emitting VC_EVENT_STREAM_STARTED/STOPPED as streams appear/disappear.
|
||||||
void sync_remote_streams(const voicecat::v1::User& user);
|
void sync_remote_streams(const voicecat::v1::User& user);
|
||||||
@@ -174,6 +215,12 @@ struct vc_client {
|
|||||||
// Joins udp_thread_, stops audio_engine_, clears media crypto/remote-stream state.
|
// Joins udp_thread_, stops audio_engine_, clears media crypto/remote-stream state.
|
||||||
// Safe to call multiple times. Called both from run_io()'s cleanup and disconnect().
|
// Safe to call multiple times. Called both from run_io()'s cleanup and disconnect().
|
||||||
void teardown_voice();
|
void teardown_voice();
|
||||||
|
// talk_timer_thread_ entry point (M3): polls audio_engine_ for remote talk-state edges
|
||||||
|
// and local capture activity, emitting VC_EVENT_TALK_STATE. Never the audio RT thread.
|
||||||
|
void run_talk_timer();
|
||||||
|
// Find a LocalStream by its client-assigned stream_id (held under local_streams_mu_ by
|
||||||
|
// the caller, or taken internally). Returns nullptr if not found/not active.
|
||||||
|
LocalStream* find_local_stream_by_id(uint32_t stream_id);
|
||||||
|
|
||||||
// ── Helpers (io_thread_ and caller threads) ─────────────────────────────────
|
// ── Helpers (io_thread_ and caller threads) ─────────────────────────────────
|
||||||
// Queue an encoded envelope to be sent on io_thread_.
|
// Queue an encoded envelope to be sent on io_thread_.
|
||||||
|
|||||||
@@ -38,6 +38,13 @@ std::vector<Stream> copy_streams(
|
|||||||
s.label = pb.label();
|
s.label = pb.label();
|
||||||
s.sample_rate = pb.audio().sample_rate() ? pb.audio().sample_rate() : 48000;
|
s.sample_rate = pb.audio().sample_rate() ? pb.audio().sample_rate() : 48000;
|
||||||
s.frame_ms = pb.audio().frame_ms() ? pb.audio().frame_ms() : 20;
|
s.frame_ms = pb.audio().frame_ms() ? pb.audio().frame_ms() : 20;
|
||||||
|
s.mode = static_cast<uint32_t>(pb.audio().mode());
|
||||||
|
s.bitrate_bps = pb.audio().bitrate_bps();
|
||||||
|
s.application = static_cast<uint32_t>(pb.audio().application());
|
||||||
|
s.fec = pb.audio().fec();
|
||||||
|
s.expected_packet_loss = pb.audio().expected_packet_loss();
|
||||||
|
s.dtx = pb.audio().dtx();
|
||||||
|
s.complexity = pb.audio().complexity();
|
||||||
out.push_back(std::move(s));
|
out.push_back(std::move(s));
|
||||||
}
|
}
|
||||||
return out;
|
return out;
|
||||||
|
|||||||
@@ -31,8 +31,19 @@ struct Stream {
|
|||||||
uint32_t ssrc{0};
|
uint32_t ssrc{0};
|
||||||
int kind{0};
|
int kind{0};
|
||||||
std::string label;
|
std::string label;
|
||||||
|
// Full effective AudioConfig (docs/protocol.md §5), as broadcast by the server in
|
||||||
|
// StreamInfo.audio — mirrors voicecat::v1::AudioConfig field-for-field so M3 per-channel
|
||||||
|
// tuning (mono/stereo, bitrate, FEC/DTX, application) is observable client-side, not just
|
||||||
|
// sample_rate/frame_ms.
|
||||||
uint32_t sample_rate{48000};
|
uint32_t sample_rate{48000};
|
||||||
uint32_t frame_ms{20};
|
uint32_t frame_ms{20};
|
||||||
|
uint32_t mode{0}; // 0 = mono, 1 = stereo (ChannelMode)
|
||||||
|
uint32_t bitrate_bps{0};
|
||||||
|
uint32_t application{0}; // 0=VOIP, 1=AUDIO, 2=LOWDELAY (OpusApplication)
|
||||||
|
bool fec{false};
|
||||||
|
uint32_t expected_packet_loss{0};
|
||||||
|
bool dtx{false};
|
||||||
|
uint32_t complexity{0};
|
||||||
};
|
};
|
||||||
|
|
||||||
struct User {
|
struct User {
|
||||||
|
|||||||
@@ -116,6 +116,18 @@ vc_result vc_set_remote_stream(vc_client* c, uint32_t user_id, uint32_t stream_i
|
|||||||
return c->set_remote_stream(user_id, stream_id, gain, muted != 0, noise_reduction != 0);
|
return c->set_remote_stream(user_id, stream_id, gain, muted != 0, noise_reduction != 0);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
vc_result vc_get_stream_audio_config(vc_client* c, uint32_t user_id, uint32_t stream_id,
|
||||||
|
vc_audio_config* out) {
|
||||||
|
if (c == nullptr || out == nullptr) return VC_ERR_INVALID_ARG;
|
||||||
|
return c->get_stream_audio_config(user_id, stream_id, out);
|
||||||
|
}
|
||||||
|
|
||||||
|
vc_result vc_test_inject_capture(vc_client* c, uint32_t stream_id, const int16_t* pcm,
|
||||||
|
size_t samples) {
|
||||||
|
if (c == nullptr || pcm == nullptr) return VC_ERR_INVALID_ARG;
|
||||||
|
return c->test_inject_capture(stream_id, pcm, samples);
|
||||||
|
}
|
||||||
|
|
||||||
vc_result vc_send_text(vc_client* c, vc_text_scope scope, uint32_t target_id,
|
vc_result vc_send_text(vc_client* c, vc_text_scope scope, uint32_t target_id,
|
||||||
const char* utf8) {
|
const char* utf8) {
|
||||||
if (c == nullptr || utf8 == nullptr) return VC_ERR_INVALID_ARG;
|
if (c == nullptr || utf8 == nullptr) return VC_ERR_INVALID_ARG;
|
||||||
|
|||||||
@@ -2,6 +2,7 @@
|
|||||||
|
|
||||||
#ifdef VOICECAT_HAS_NET
|
#ifdef VOICECAT_HAS_NET
|
||||||
|
|
||||||
|
#include <algorithm>
|
||||||
#include <chrono>
|
#include <chrono>
|
||||||
#include <cstdio>
|
#include <cstdio>
|
||||||
#include <cstring>
|
#include <cstring>
|
||||||
@@ -329,15 +330,31 @@ void ConnSession::handle_udp_binding(uint64_t req_id, const voicecat::v1::UdpBin
|
|||||||
void ConnSession::handle_stream_announce(uint64_t req_id,
|
void ConnSession::handle_stream_announce(uint64_t req_id,
|
||||||
const voicecat::v1::StreamAnnounce& msg) {
|
const voicecat::v1::StreamAnnounce& msg) {
|
||||||
uint32_t ssrc = registry_->assign_ssrc(session_id_);
|
uint32_t ssrc = registry_->assign_ssrc(session_id_);
|
||||||
|
uint32_t stream_id = next_stream_id_++;
|
||||||
|
|
||||||
auto env = make_env(req_id);
|
auto env = make_env(req_id);
|
||||||
auto* res = env.mutable_stream_announce_result();
|
auto* res = env.mutable_stream_announce_result();
|
||||||
res->set_ok(true);
|
res->set_ok(true);
|
||||||
res->set_stream_id(1);
|
res->set_stream_id(stream_id);
|
||||||
res->set_ssrc(ssrc);
|
res->set_ssrc(ssrc);
|
||||||
|
|
||||||
|
// Per-channel AudioConfig is authoritative (docs/voice.md §3): the channel's mode/
|
||||||
|
// frame_ms/application/fec/dtx/complexity/expected_packet_loss apply to every stream
|
||||||
|
// announced into it, regardless of kind. bitrate_bps is clamped (not overridden) to the
|
||||||
|
// channel's ceiling so a client may still request less. sample_rate stays
|
||||||
|
// client-requested-or-48000 — everything runs at 48kHz internally per voice.md §3.
|
||||||
auto* eff = res->mutable_effective_audio();
|
auto* eff = res->mutable_effective_audio();
|
||||||
if (msg.has_requested_audio()) {
|
auto chan_cfg = registry_->channel_audio_config(registry_->user_channel(user_id_.load()));
|
||||||
|
uint32_t requested_bps =
|
||||||
|
msg.has_requested_audio() ? msg.requested_audio().bitrate_bps() : 0;
|
||||||
|
uint32_t requested_rate =
|
||||||
|
msg.has_requested_audio() ? msg.requested_audio().sample_rate() : 0;
|
||||||
|
if (chan_cfg) {
|
||||||
|
*eff = *chan_cfg;
|
||||||
|
eff->set_bitrate_bps(requested_bps > 0 ? std::min(requested_bps, chan_cfg->bitrate_bps())
|
||||||
|
: chan_cfg->bitrate_bps());
|
||||||
|
eff->set_sample_rate(requested_rate > 0 ? requested_rate : 48000);
|
||||||
|
} else if (msg.has_requested_audio()) {
|
||||||
*eff = msg.requested_audio();
|
*eff = msg.requested_audio();
|
||||||
} else {
|
} else {
|
||||||
eff->set_codec(0); // OPUS
|
eff->set_codec(0); // OPUS
|
||||||
@@ -350,10 +367,10 @@ void ConnSession::handle_stream_announce(uint64_t req_id,
|
|||||||
if (eff->bitrate_bps() == 0) eff->set_bitrate_bps(24000);
|
if (eff->bitrate_bps() == 0) eff->set_bitrate_bps(24000);
|
||||||
if (eff->frame_ms() == 0) eff->set_frame_ms(20);
|
if (eff->frame_ms() == 0) eff->set_frame_ms(20);
|
||||||
|
|
||||||
announced_stream_id_ = res->stream_id();
|
announced_stream_ids_.push_back(stream_id);
|
||||||
|
|
||||||
voicecat::v1::StreamInfo info;
|
voicecat::v1::StreamInfo info;
|
||||||
info.set_stream_id(res->stream_id());
|
info.set_stream_id(stream_id);
|
||||||
info.set_ssrc(ssrc);
|
info.set_ssrc(ssrc);
|
||||||
info.set_kind(msg.kind());
|
info.set_kind(msg.kind());
|
||||||
*info.mutable_audio() = *eff;
|
*info.mutable_audio() = *eff;
|
||||||
@@ -372,8 +389,12 @@ void ConnSession::handle_stream_announce(uint64_t req_id,
|
|||||||
}
|
}
|
||||||
|
|
||||||
void ConnSession::handle_stream_stop(const voicecat::v1::StreamStop& msg) {
|
void ConnSession::handle_stream_stop(const voicecat::v1::StreamStop& msg) {
|
||||||
|
auto it = std::find(announced_stream_ids_.begin(), announced_stream_ids_.end(),
|
||||||
|
msg.stream_id());
|
||||||
|
if (it == announced_stream_ids_.end()) return; // not ours — ignore (no spoofed stops)
|
||||||
|
announced_stream_ids_.erase(it);
|
||||||
|
|
||||||
auto updated = registry_->clear_user_stream(user_id_.load(), msg.stream_id());
|
auto updated = registry_->clear_user_stream(user_id_.load(), msg.stream_id());
|
||||||
if (msg.stream_id() == announced_stream_id_) announced_stream_id_ = 0;
|
|
||||||
if (updated) {
|
if (updated) {
|
||||||
auto bcast = make_env();
|
auto bcast = make_env();
|
||||||
auto* ue = bcast.mutable_user_event();
|
auto* ue = bcast.mutable_user_event();
|
||||||
|
|||||||
@@ -121,8 +121,11 @@ class ConnSession : public std::enable_shared_from_this<ConnSession> {
|
|||||||
std::unique_ptr<voicecat::crypto::SodiumMediaCrypto> send_crypto_;
|
std::unique_ptr<voicecat::crypto::SodiumMediaCrypto> send_crypto_;
|
||||||
std::unique_ptr<voicecat::crypto::SodiumMediaCrypto> recv_crypto_;
|
std::unique_ptr<voicecat::crypto::SodiumMediaCrypto> recv_crypto_;
|
||||||
|
|
||||||
// M2: locally-announced stream (single MIC stream per session for now)
|
// M2/M3: locally-announced streams. The server assigns the stream_id (unique per
|
||||||
uint32_t announced_stream_id_{0};
|
// session), so a per-session counter + the set of currently-active ids is enough to
|
||||||
|
// support multiple concurrent streams (MIC + SCREEN_AUDIO + AUX_DEVICE) per user.
|
||||||
|
uint32_t next_stream_id_{1};
|
||||||
|
std::vector<uint32_t> announced_stream_ids_;
|
||||||
};
|
};
|
||||||
|
|
||||||
} // namespace voicecat::server
|
} // namespace voicecat::server
|
||||||
|
|||||||
@@ -17,7 +17,44 @@ void SessionRegistry::init_default_channels() {
|
|||||||
lobby.proto.set_name("Lobby");
|
lobby.proto.set_name("Lobby");
|
||||||
lobby.proto.set_type(voicecat::v1::CHANNEL_PERMANENT);
|
lobby.proto.set_type(voicecat::v1::CHANNEL_PERMANENT);
|
||||||
lobby.proto.set_order(0);
|
lobby.proto.set_order(0);
|
||||||
|
{
|
||||||
|
// Speech profile: mono, low bitrate, FEC+DTX on for resilience/silence-suppression.
|
||||||
|
auto* a = lobby.proto.mutable_audio();
|
||||||
|
a->set_codec(0);
|
||||||
|
a->set_mode(voicecat::v1::MODE_MONO);
|
||||||
|
a->set_sample_rate(48000);
|
||||||
|
a->set_bitrate_bps(24000);
|
||||||
|
a->set_frame_ms(20);
|
||||||
|
a->set_application(voicecat::v1::OPUS_VOIP);
|
||||||
|
a->set_fec(true);
|
||||||
|
a->set_expected_packet_loss(10);
|
||||||
|
a->set_dtx(true);
|
||||||
|
a->set_complexity(5);
|
||||||
|
}
|
||||||
channels_[1] = std::move(lobby);
|
channels_[1] = std::move(lobby);
|
||||||
|
|
||||||
|
ChannelEntry music;
|
||||||
|
music.proto.set_id(2);
|
||||||
|
music.proto.set_name("Music Room");
|
||||||
|
music.proto.set_type(voicecat::v1::CHANNEL_PERMANENT);
|
||||||
|
music.proto.set_order(1);
|
||||||
|
{
|
||||||
|
// Music/screen-audio profile: stereo, high bitrate, FEC/DTX off (continuous signal).
|
||||||
|
auto* a = music.proto.mutable_audio();
|
||||||
|
a->set_codec(0);
|
||||||
|
a->set_mode(voicecat::v1::MODE_STEREO);
|
||||||
|
a->set_sample_rate(48000);
|
||||||
|
a->set_bitrate_bps(128000);
|
||||||
|
a->set_frame_ms(20);
|
||||||
|
a->set_application(voicecat::v1::OPUS_AUDIO);
|
||||||
|
a->set_fec(false);
|
||||||
|
a->set_expected_packet_loss(0);
|
||||||
|
a->set_dtx(false);
|
||||||
|
a->set_complexity(8);
|
||||||
|
}
|
||||||
|
channels_[2] = std::move(music);
|
||||||
|
|
||||||
|
next_channel_id_ = 3; // 1 and 2 are now reserved (Lobby, Music Room)
|
||||||
}
|
}
|
||||||
|
|
||||||
uint64_t SessionRegistry::register_session(std::weak_ptr<ConnSession> session) {
|
uint64_t SessionRegistry::register_session(std::weak_ptr<ConnSession> session) {
|
||||||
@@ -203,6 +240,14 @@ uint32_t SessionRegistry::user_channel(uint32_t user_id) const {
|
|||||||
return (it == users_.end()) ? 0 : it->second.proto.channel_id();
|
return (it == users_.end()) ? 0 : it->second.proto.channel_id();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
std::optional<voicecat::v1::AudioConfig> SessionRegistry::channel_audio_config(
|
||||||
|
uint32_t channel_id) const {
|
||||||
|
std::shared_lock lk(mu_);
|
||||||
|
auto it = channels_.find(channel_id);
|
||||||
|
if (it == channels_.end()) return std::nullopt;
|
||||||
|
return it->second.proto.audio();
|
||||||
|
}
|
||||||
|
|
||||||
} // namespace voicecat::server
|
} // namespace voicecat::server
|
||||||
|
|
||||||
#endif // VOICECAT_HAS_NET
|
#endif // VOICECAT_HAS_NET
|
||||||
|
|||||||
@@ -112,6 +112,12 @@ class SessionRegistry {
|
|||||||
// Return the channel_id of a user (0 if not found).
|
// Return the channel_id of a user (0 if not found).
|
||||||
uint32_t user_channel(uint32_t user_id) const;
|
uint32_t user_channel(uint32_t user_id) const;
|
||||||
|
|
||||||
|
// Return a channel's authoritative AudioConfig (M3 per-channel Opus tuning), or nullopt
|
||||||
|
// if the channel doesn't exist. There is no per-id Channel getter today otherwise —
|
||||||
|
// channel_snapshot() copies every channel, which callers needing just one config should
|
||||||
|
// avoid.
|
||||||
|
std::optional<voicecat::v1::AudioConfig> channel_audio_config(uint32_t channel_id) const;
|
||||||
|
|
||||||
private:
|
private:
|
||||||
mutable std::shared_mutex mu_;
|
mutable std::shared_mutex mu_;
|
||||||
|
|
||||||
|
|||||||
@@ -76,4 +76,14 @@ if(VOICECAT_USE_VCPKG_DEPS)
|
|||||||
target_include_directories(test_voice_client_abi PRIVATE ${VC_TEST_INTERNAL_INCLUDES})
|
target_include_directories(test_voice_client_abi PRIVATE ${VC_TEST_INTERNAL_INCLUDES})
|
||||||
add_test(NAME voice_client_abi COMMAND test_voice_client_abi)
|
add_test(NAME voice_client_abi COMMAND test_voice_client_abi)
|
||||||
set_tests_properties(voice_client_abi PROPERTIES TIMEOUT 60)
|
set_tests_properties(voice_client_abi PROPERTIES TIMEOUT 60)
|
||||||
|
|
||||||
|
# M3 exit criterion: multi-stream (mic + desktop audio), independent per-stream
|
||||||
|
# gain/mute/NS, per-channel Opus configurability, talk indicators -- all through the
|
||||||
|
# real C ABI (vc_client), not raw sockets.
|
||||||
|
add_executable(test_m3_multistream test_m3_multistream.cpp)
|
||||||
|
target_link_libraries(test_m3_multistream PRIVATE voicecat::server)
|
||||||
|
target_compile_features(test_m3_multistream PRIVATE cxx_std_20)
|
||||||
|
target_include_directories(test_m3_multistream PRIVATE ${VC_TEST_INTERNAL_INCLUDES})
|
||||||
|
add_test(NAME m3_multistream COMMAND test_m3_multistream)
|
||||||
|
set_tests_properties(m3_multistream PROPERTIES TIMEOUT 60)
|
||||||
endif()
|
endif()
|
||||||
|
|||||||
356
tests/test_m3_multistream.cpp
Normal file
356
tests/test_m3_multistream.cpp
Normal file
@@ -0,0 +1,356 @@
|
|||||||
|
/*
|
||||||
|
* test_m3_multistream — M3 exit criterion, exercised through the real C ABI.
|
||||||
|
*
|
||||||
|
* Mirrors test_voice_client_abi.cpp's approach (real vc_client instances, not raw sockets —
|
||||||
|
* the M2 lesson is that ABI-level coverage is what actually proves the client library works).
|
||||||
|
* Covers the whole M3 milestone in one flow:
|
||||||
|
*
|
||||||
|
* 1. A starts two concurrent local streams (MIC + SCREEN_AUDIO) -- distinct stream ids,
|
||||||
|
* both visible to B as separate STREAM_STARTED events for the same user.
|
||||||
|
* 2. Synthetic PCM (vc_test_inject_capture) flows into both of A's streams without crashing
|
||||||
|
* and without disrupting the control/voice plane; B observes a VC_EVENT_TALK_STATE
|
||||||
|
* talking=true edge for A's MIC stream while both are still in the same channel (voice
|
||||||
|
* only relays within a channel, so this must happen before step 4 moves A elsewhere).
|
||||||
|
* 3. B independently gains/mutes/NS-toggles A's two streams (vc_set_remote_stream) --
|
||||||
|
* one call doesn't clobber the other's routing; a bogus stream_id is rejected.
|
||||||
|
* 4. Per-channel Opus configurability: A joins "Music Room" (channel 2, stereo/128kbps/
|
||||||
|
* OPUS_AUDIO/no DTX) before announcing there, while B stays in "Lobby" (channel 1,
|
||||||
|
* mono/24kbps/OPUS_VOIP/DTX) -- vc_get_stream_audio_config shows the two streams'
|
||||||
|
* effective config differs exactly as the server enforces it.
|
||||||
|
*/
|
||||||
|
#include <cstdio>
|
||||||
|
|
||||||
|
#ifdef VOICECAT_HAS_NET
|
||||||
|
|
||||||
|
#include <atomic>
|
||||||
|
#include <chrono>
|
||||||
|
#include <cmath>
|
||||||
|
#include <condition_variable>
|
||||||
|
#include <filesystem>
|
||||||
|
#include <mutex>
|
||||||
|
#include <string>
|
||||||
|
#include <thread>
|
||||||
|
#include <vector>
|
||||||
|
|
||||||
|
#include "voicecat.h"
|
||||||
|
#include "server.h"
|
||||||
|
#include "db.h"
|
||||||
|
|
||||||
|
// ── Event tracking ────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
struct StreamEvent {
|
||||||
|
bool started; // true = STARTED, false = STOPPED
|
||||||
|
uint32_t user_id;
|
||||||
|
uint32_t stream_id;
|
||||||
|
};
|
||||||
|
|
||||||
|
struct TalkEvent {
|
||||||
|
uint32_t user_id;
|
||||||
|
uint32_t stream_id;
|
||||||
|
bool talking;
|
||||||
|
};
|
||||||
|
|
||||||
|
struct EventStore {
|
||||||
|
std::mutex mu;
|
||||||
|
std::condition_variable cv;
|
||||||
|
|
||||||
|
bool auth_ok{false};
|
||||||
|
uint32_t self_user_id{0};
|
||||||
|
bool channel_list_received{false};
|
||||||
|
std::vector<StreamEvent> stream_events;
|
||||||
|
std::vector<TalkEvent> talk_events;
|
||||||
|
bool disconnected{false};
|
||||||
|
|
||||||
|
const char* label{nullptr};
|
||||||
|
};
|
||||||
|
|
||||||
|
static void on_event(void* user, const vc_event* ev) {
|
||||||
|
auto* s = static_cast<EventStore*>(user);
|
||||||
|
std::lock_guard lk(s->mu);
|
||||||
|
switch (ev->type) {
|
||||||
|
case VC_EVENT_AUTH_RESULT:
|
||||||
|
s->auth_ok = (ev->result == VC_OK);
|
||||||
|
s->self_user_id = ev->user_id;
|
||||||
|
if (!s->auth_ok) std::fprintf(stderr, "[%s] AUTH FAILED: %s\n",
|
||||||
|
s->label ? s->label : "?", ev->text ? ev->text : "(no msg)");
|
||||||
|
break;
|
||||||
|
case VC_EVENT_CHANNEL_LIST:
|
||||||
|
s->channel_list_received = true;
|
||||||
|
break;
|
||||||
|
case VC_EVENT_STREAM_STARTED:
|
||||||
|
s->stream_events.push_back({true, ev->user_id, ev->stream_id});
|
||||||
|
break;
|
||||||
|
case VC_EVENT_STREAM_STOPPED:
|
||||||
|
s->stream_events.push_back({false, ev->user_id, ev->stream_id});
|
||||||
|
break;
|
||||||
|
case VC_EVENT_TALK_STATE:
|
||||||
|
s->talk_events.push_back({ev->user_id, ev->stream_id, ev->u32a != 0});
|
||||||
|
break;
|
||||||
|
case VC_EVENT_ERROR:
|
||||||
|
std::fprintf(stderr, "[%s] ERROR rc=%d: %s\n",
|
||||||
|
s->label ? s->label : "?", ev->result, ev->text ? ev->text : "");
|
||||||
|
break;
|
||||||
|
case VC_EVENT_DISCONNECTED:
|
||||||
|
s->disconnected = true;
|
||||||
|
break;
|
||||||
|
default:
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
s->cv.notify_all();
|
||||||
|
}
|
||||||
|
|
||||||
|
template <typename Pred>
|
||||||
|
static bool wait_for(EventStore& s, Pred pred, int timeout_ms) {
|
||||||
|
auto deadline = std::chrono::steady_clock::now() + std::chrono::milliseconds(timeout_ms);
|
||||||
|
std::unique_lock lk(s.mu);
|
||||||
|
return s.cv.wait_until(lk, deadline, [&] { return pred(s); });
|
||||||
|
}
|
||||||
|
|
||||||
|
static std::vector<int16_t> make_sine_frame(int frame_idx, float freq_hz,
|
||||||
|
int frame_samples = 960) {
|
||||||
|
std::vector<int16_t> pcm(frame_samples);
|
||||||
|
for (int i = 0; i < frame_samples; ++i) {
|
||||||
|
float t = static_cast<float>(frame_idx * frame_samples + i) / 48000.0f;
|
||||||
|
pcm[i] = static_cast<int16_t>(std::sin(2.0f * 3.14159265f * freq_hz * t) * 16000.0f);
|
||||||
|
}
|
||||||
|
return pcm;
|
||||||
|
}
|
||||||
|
|
||||||
|
// ── Test harness ──────────────────────────────────────────────────────────────
|
||||||
|
|
||||||
|
static int g_failures = 0;
|
||||||
|
#define CHECK(cond) \
|
||||||
|
do { \
|
||||||
|
if (!(cond)) { \
|
||||||
|
std::printf("FAIL: %s (%s:%d)\n", #cond, __FILE__, __LINE__); \
|
||||||
|
++g_failures; \
|
||||||
|
} \
|
||||||
|
} while (0)
|
||||||
|
|
||||||
|
int main() {
|
||||||
|
auto tmp = std::filesystem::temp_directory_path() /
|
||||||
|
("vctest_m3_" + std::to_string(
|
||||||
|
std::chrono::steady_clock::now().time_since_epoch().count()));
|
||||||
|
std::filesystem::create_directories(tmp);
|
||||||
|
std::string data_dir = tmp.string();
|
||||||
|
|
||||||
|
std::atomic<uint16_t> bound_port{0};
|
||||||
|
std::mutex ready_mu;
|
||||||
|
std::condition_variable ready_cv;
|
||||||
|
bool ready{false};
|
||||||
|
|
||||||
|
voicecat::server::Config cfg;
|
||||||
|
cfg.data_dir = data_dir;
|
||||||
|
cfg.bind_port = 0;
|
||||||
|
cfg.media_port = 0;
|
||||||
|
cfg.server_name = "VoiceCat-M3Test";
|
||||||
|
cfg.allow_guests = true;
|
||||||
|
cfg.on_ready = [&](uint16_t p) {
|
||||||
|
bound_port.store(p);
|
||||||
|
{ std::lock_guard lk(ready_mu); ready = true; }
|
||||||
|
ready_cv.notify_all();
|
||||||
|
};
|
||||||
|
|
||||||
|
voicecat::server::Server server(cfg);
|
||||||
|
std::thread server_thread([&] { server.run(); });
|
||||||
|
|
||||||
|
{
|
||||||
|
std::unique_lock lk(ready_mu);
|
||||||
|
bool ok = ready_cv.wait_for(lk, std::chrono::seconds(10), [&] { return ready; });
|
||||||
|
if (!ok) {
|
||||||
|
std::printf("FAIL: server did not become ready within 10s\n");
|
||||||
|
server.stop();
|
||||||
|
server_thread.join();
|
||||||
|
std::filesystem::remove_all(tmp);
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
uint16_t port = bound_port.load();
|
||||||
|
std::printf("m3_multistream: server ready on :%u\n", port);
|
||||||
|
|
||||||
|
// ── Client A: guest "M3-A" ────────────────────────────────────────────────
|
||||||
|
EventStore evA;
|
||||||
|
evA.label = "clientA";
|
||||||
|
vc_callbacks cbA{on_event, nullptr, &evA};
|
||||||
|
vc_config cfgA{"test-clientA", "0.1", VC_LOG_OFF};
|
||||||
|
vc_client* clientA = vc_client_create(&cfgA, cbA);
|
||||||
|
CHECK(clientA != nullptr);
|
||||||
|
|
||||||
|
CHECK(vc_connect(clientA, "127.0.0.1", port) == VC_OK);
|
||||||
|
CHECK(vc_authenticate_guest(clientA, "M3-A") == VC_OK);
|
||||||
|
CHECK(wait_for(evA, [](EventStore& s) { return s.auth_ok; }, 8000));
|
||||||
|
CHECK(wait_for(evA, [](EventStore& s) { return s.channel_list_received; }, 3000));
|
||||||
|
|
||||||
|
// ── Client B: guest "M3-B" ────────────────────────────────────────────────
|
||||||
|
EventStore evB;
|
||||||
|
evB.label = "clientB";
|
||||||
|
vc_callbacks cbB{on_event, nullptr, &evB};
|
||||||
|
vc_config cfgB{"test-clientB", "0.1", VC_LOG_OFF};
|
||||||
|
vc_client* clientB = vc_client_create(&cfgB, cbB);
|
||||||
|
CHECK(clientB != nullptr);
|
||||||
|
|
||||||
|
CHECK(vc_connect(clientB, "127.0.0.1", port) == VC_OK);
|
||||||
|
CHECK(vc_authenticate_guest(clientB, "M3-B") == VC_OK);
|
||||||
|
CHECK(wait_for(evB, [](EventStore& s) { return s.auth_ok; }, 8000));
|
||||||
|
CHECK(wait_for(evB, [](EventStore& s) { return s.channel_list_received; }, 3000));
|
||||||
|
|
||||||
|
uint32_t a_uid = 0;
|
||||||
|
{ std::lock_guard lk(evA.mu); a_uid = evA.self_user_id; }
|
||||||
|
|
||||||
|
// Both guests land in channel 1 (Lobby) automatically; give the async UDP binding
|
||||||
|
// handshake a moment to complete on both clients before announcing streams.
|
||||||
|
std::this_thread::sleep_for(std::chrono::milliseconds(500));
|
||||||
|
|
||||||
|
// ── 1. A starts MIC + SCREEN_AUDIO concurrently ──────────────────────────
|
||||||
|
vc_stream_desc mic_desc{};
|
||||||
|
mic_desc.kind = VC_STREAM_MIC;
|
||||||
|
mic_desc.label = "mic";
|
||||||
|
uint32_t mic_sid = 0;
|
||||||
|
CHECK(vc_stream_start(clientA, &mic_desc, &mic_sid) == VC_OK);
|
||||||
|
|
||||||
|
vc_stream_desc screen_desc{};
|
||||||
|
screen_desc.kind = VC_STREAM_SCREEN_AUDIO;
|
||||||
|
screen_desc.label = "desktop audio";
|
||||||
|
uint32_t screen_sid = 0;
|
||||||
|
CHECK(vc_stream_start(clientA, &screen_desc, &screen_sid) == VC_OK);
|
||||||
|
|
||||||
|
CHECK(mic_sid != 0 && screen_sid != 0 && mic_sid != screen_sid);
|
||||||
|
|
||||||
|
// B observes two distinct STREAM_STARTED events for user A.
|
||||||
|
bool b_saw_both = wait_for(evB, [&](EventStore& s) {
|
||||||
|
bool saw_mic = false, saw_screen = false;
|
||||||
|
for (auto& e : s.stream_events) {
|
||||||
|
if (!e.started || e.user_id != a_uid) continue;
|
||||||
|
if (e.stream_id == mic_sid) saw_mic = true;
|
||||||
|
if (e.stream_id == screen_sid) saw_screen = true;
|
||||||
|
}
|
||||||
|
return saw_mic && saw_screen;
|
||||||
|
}, 5000);
|
||||||
|
CHECK(b_saw_both);
|
||||||
|
|
||||||
|
// Also wait for A's own view of both streams (vc_test_inject_capture requires the
|
||||||
|
// LocalStream to be active, which flips on A's io_thread_ independently of -- and not
|
||||||
|
// necessarily before -- the broadcast B observes above).
|
||||||
|
bool a_self_saw_both = wait_for(evA, [&](EventStore& s) {
|
||||||
|
bool saw_mic = false, saw_screen = false;
|
||||||
|
for (auto& e : s.stream_events) {
|
||||||
|
if (!e.started || e.user_id != a_uid) continue;
|
||||||
|
if (e.stream_id == mic_sid) saw_mic = true;
|
||||||
|
if (e.stream_id == screen_sid) saw_screen = true;
|
||||||
|
}
|
||||||
|
return saw_mic && saw_screen;
|
||||||
|
}, 5000);
|
||||||
|
CHECK(a_self_saw_both);
|
||||||
|
|
||||||
|
// ── 2. Inject synthetic PCM into both of A's local streams ──────────────
|
||||||
|
for (int i = 0; i < 25; ++i) {
|
||||||
|
auto mic_pcm = make_sine_frame(i, 440.0f);
|
||||||
|
auto screen_pcm = make_sine_frame(i, 880.0f);
|
||||||
|
CHECK(vc_test_inject_capture(clientA, mic_sid, mic_pcm.data(), mic_pcm.size()) == VC_OK);
|
||||||
|
CHECK(vc_test_inject_capture(clientA, screen_sid, screen_pcm.data(), screen_pcm.size()) == VC_OK);
|
||||||
|
std::this_thread::sleep_for(std::chrono::milliseconds(20));
|
||||||
|
}
|
||||||
|
|
||||||
|
// No disconnects/errors should have resulted from the dual-stream PCM flow.
|
||||||
|
{ std::lock_guard lk(evA.mu); CHECK(!evA.disconnected); }
|
||||||
|
{ std::lock_guard lk(evB.mu); CHECK(!evB.disconnected); }
|
||||||
|
|
||||||
|
// ── 5. Talk indicators ────────────────────────────────────────────────────
|
||||||
|
// While A and B are still both in Lobby (voice actually relays between them here --
|
||||||
|
// the SFU forwards within a channel, so this must happen before A moves to Music Room
|
||||||
|
// in step 4 below), confirm B observed a talking=true edge for A's MIC stream.
|
||||||
|
bool b_saw_talking = wait_for(evB, [&](EventStore& s) {
|
||||||
|
for (auto& e : s.talk_events)
|
||||||
|
if (e.user_id == a_uid && e.stream_id == mic_sid && e.talking) return true;
|
||||||
|
return false;
|
||||||
|
}, 3000);
|
||||||
|
CHECK(b_saw_talking);
|
||||||
|
|
||||||
|
// ── 3. B independently controls gain/mute/NS on each of A's streams ─────
|
||||||
|
CHECK(vc_set_remote_stream(clientB, a_uid, mic_sid, 1.0f, 0, 0) == VC_OK);
|
||||||
|
CHECK(vc_set_remote_stream(clientB, a_uid, screen_sid, 0.3f, 1, 1) == VC_OK);
|
||||||
|
CHECK(vc_set_remote_stream(clientB, a_uid, 0xDEADBEEF, 1.0f, 0, 0) == VC_ERR_INVALID_ARG);
|
||||||
|
|
||||||
|
// Toggle NS on/off a few times -- plumbing should never fault or disrupt the stream.
|
||||||
|
for (int i = 0; i < 3; ++i) {
|
||||||
|
CHECK(vc_set_remote_stream(clientB, a_uid, mic_sid, 1.0f, 0, 1) == VC_OK);
|
||||||
|
CHECK(vc_set_remote_stream(clientB, a_uid, mic_sid, 1.0f, 0, 0) == VC_OK);
|
||||||
|
}
|
||||||
|
{ std::lock_guard lk(evB.mu); CHECK(!evB.disconnected); }
|
||||||
|
|
||||||
|
// ── 4. Per-channel Opus configurability ──────────────────────────────────
|
||||||
|
// A moves to "Music Room" (channel 2: stereo/128kbps/OPUS_AUDIO/no DTX) and announces a
|
||||||
|
// fresh MIC stream there; B stays in "Lobby" (channel 1: mono/24kbps/OPUS_VOIP/DTX) with
|
||||||
|
// its own MIC stream. Their effective_audio should differ exactly as configured server-side.
|
||||||
|
CHECK(vc_stream_stop(clientA, mic_sid) == VC_OK);
|
||||||
|
CHECK(vc_join_channel(clientA, 2, nullptr) == VC_OK);
|
||||||
|
std::this_thread::sleep_for(std::chrono::milliseconds(300));
|
||||||
|
|
||||||
|
vc_stream_desc music_mic_desc{};
|
||||||
|
music_mic_desc.kind = VC_STREAM_MIC;
|
||||||
|
music_mic_desc.label = "music-mic";
|
||||||
|
uint32_t a_music_mic_sid = 0;
|
||||||
|
CHECK(vc_stream_start(clientA, &music_mic_desc, &a_music_mic_sid) == VC_OK);
|
||||||
|
CHECK(wait_for(evA, [&](EventStore& s) {
|
||||||
|
for (auto& e : s.stream_events)
|
||||||
|
if (e.started && e.user_id == a_uid && e.stream_id == a_music_mic_sid) return true;
|
||||||
|
return false;
|
||||||
|
}, 5000));
|
||||||
|
|
||||||
|
vc_stream_desc b_mic_desc{};
|
||||||
|
b_mic_desc.kind = VC_STREAM_MIC;
|
||||||
|
b_mic_desc.label = "lobby-mic";
|
||||||
|
uint32_t b_mic_sid = 0;
|
||||||
|
CHECK(vc_stream_start(clientB, &b_mic_desc, &b_mic_sid) == VC_OK);
|
||||||
|
CHECK(wait_for(evB, [&](EventStore& s) {
|
||||||
|
uint32_t self = s.self_user_id;
|
||||||
|
for (auto& e : s.stream_events)
|
||||||
|
if (e.started && e.user_id == self && e.stream_id == b_mic_sid) return true;
|
||||||
|
return false;
|
||||||
|
}, 5000));
|
||||||
|
|
||||||
|
vc_audio_config a_cfg{};
|
||||||
|
vc_audio_config b_cfg{};
|
||||||
|
CHECK(vc_get_stream_audio_config(clientA, a_uid, a_music_mic_sid, &a_cfg) == VC_OK);
|
||||||
|
uint32_t b_uid = 0;
|
||||||
|
{ std::lock_guard lk(evB.mu); b_uid = evB.self_user_id; }
|
||||||
|
CHECK(vc_get_stream_audio_config(clientB, b_uid, b_mic_sid, &b_cfg) == VC_OK);
|
||||||
|
|
||||||
|
// Music Room: stereo, 128kbps, OPUS_AUDIO, DTX off. Lobby: mono, 24kbps, OPUS_VOIP, DTX on.
|
||||||
|
CHECK(a_cfg.mode == 1 /* stereo */);
|
||||||
|
CHECK(b_cfg.mode == 0 /* mono */);
|
||||||
|
CHECK(a_cfg.bitrate_bps == 128000);
|
||||||
|
CHECK(b_cfg.bitrate_bps == 24000);
|
||||||
|
CHECK(a_cfg.application == 1 /* OPUS_AUDIO */);
|
||||||
|
CHECK(b_cfg.application == 0 /* OPUS_VOIP */);
|
||||||
|
CHECK(a_cfg.dtx == 0);
|
||||||
|
CHECK(b_cfg.dtx != 0);
|
||||||
|
|
||||||
|
// ── Cleanup ───────────────────────────────────────────────────────────────
|
||||||
|
vc_disconnect(clientA);
|
||||||
|
vc_disconnect(clientB);
|
||||||
|
vc_client_destroy(clientA);
|
||||||
|
vc_client_destroy(clientB);
|
||||||
|
|
||||||
|
server.stop();
|
||||||
|
server_thread.join();
|
||||||
|
|
||||||
|
std::filesystem::remove_all(tmp);
|
||||||
|
|
||||||
|
if (g_failures == 0) {
|
||||||
|
std::printf("m3_multistream: all checks passed\n");
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
std::printf("m3_multistream: %d failure(s)\n", g_failures);
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
#else // !VOICECAT_HAS_NET
|
||||||
|
|
||||||
|
int main() {
|
||||||
|
std::printf("m3_multistream: SKIP (VOICECAT_HAS_NET not defined)\n");
|
||||||
|
return 0;
|
||||||
|
}
|
||||||
|
|
||||||
|
#endif // VOICECAT_HAS_NET
|
||||||
Reference in New Issue
Block a user