fix(audio): reframe send path to channel frame_ms; pin codec to 48 kHz
The AudioEngine capture clock is fixed at 48 kHz / 20 ms (960-sample frames), but a channel may set any Opus frame_ms (2.5..60 ms, voice.md §3) and the server enforces it unclamped. on_capture_frame handed the engine's 960-sample frame straight to an encoder configured for the channel's window: frame_ms > 20 was silently ignored, and frame_ms < 20 broke entirely (receiver sized its decode buffer too small -> OPUS_BUFFER_TOO_SMALL -> dead audio). Affected the hardware mic and vc_stream_feed_pcm alike. Reframe each captured/fed block to ls.frame_samples via a per-LocalStream accumulator (pre-sized at announce, no RT-thread alloc) before encode_and_send_frame; the 20 ms case stays a zero-copy fast path. Also pin the codec to 48 kHz in opus_params_from_audio_config — it was honoring a non-48k effective sample_rate against a 48 kHz PCM clock. New ctest frame_ms_reframe covers 40 ms (accumulate) and 10 ms (split) feed->encode->relay->decode->sink round trips. ctest --preset dev 25/25. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -8,7 +8,7 @@ and what's next* read [`PROGRESS.md`](PROGRESS.md); for *design* read [`docs/`](
|
||||
> server-mute, channel CRUD, in-app account management, disconnect/keepalive/reaper. Windows
|
||||
> WinForms C# client shipped (M4). **macOS AppKit client shipped** — `VoiceCatMac.xcodeproj`
|
||||
> at `clients/apple/macOS/`. **iOS SwiftUI client shipped** — `VoiceCatiOS.xcodeproj` at
|
||||
> `clients/apple/iOS/`. `ctest --preset dev` green — 24/24 tests.
|
||||
> `clients/apple/iOS/`. `ctest --preset dev` green — 25/25 tests.
|
||||
> External PCM feed/tap API (`vc_stream_feed_pcm` + `vc_set_pcm_sink`) shipped.
|
||||
> **Screen-audio sharing shipped on macOS (ScreenCaptureKit) and iOS (ReplayKit Broadcast
|
||||
> Upload Extension → host App Group ring → `vc_stream_feed_pcm`).** See [`PROGRESS.md`](PROGRESS.md).
|
||||
|
||||
16
PROGRESS.md
16
PROGRESS.md
@@ -442,7 +442,21 @@ up instantly. Newest status at the top.
|
||||
|
||||
## Recent completed work
|
||||
|
||||
All items below are `[x]` done; `ctest --preset dev` 23/23 on Windows after all.
|
||||
All items below are `[x]` done; `ctest --preset dev` 25/25 on Windows after all.
|
||||
|
||||
- **Non-20ms channel frame_ms fix** (2026-06-22): the AudioEngine capture clock is fixed at
|
||||
48 kHz / 20 ms (960-sample frames), but a channel may set any Opus `frame_ms` (2.5…60 ms,
|
||||
docs/voice.md §3) and the server enforces it unclamped. The send path handed the engine's
|
||||
960-sample frame straight to an encoder configured for the channel's window — silently
|
||||
ignoring `frame_ms > 20` and **breaking `frame_ms < 20` entirely** (receiver sized its decode
|
||||
buffer too small → `OPUS_BUFFER_TOO_SMALL` → dead audio). Affected the hardware mic AND
|
||||
`vc_stream_feed_pcm`. Fix: `vc_client::on_capture_frame` now reframes each captured/fed block
|
||||
to `ls.frame_samples` via a per-`LocalStream` accumulator (pre-sized at announce, no RT-thread
|
||||
alloc) before `encode_and_send_frame`; the 20 ms case stays a zero-copy fast path. Also pinned
|
||||
the codec to 48 kHz internally in `opus_params_from_audio_config` (was honoring a non-48k
|
||||
effective sample_rate against a 48k PCM clock). New ctest `frame_ms_reframe` (40 ms accumulate
|
||||
+ 10 ms split round trips). Files: `client.{h,cpp}`, `voicecat.h` (feed doc), `docs/voice.md`,
|
||||
`tests/test_frame_ms_reframe.cpp`, `tests/CMakeLists.txt`.
|
||||
|
||||
- **External PCM feed/tap API** (2026-06-20): `vc_stream_feed_pcm` + `vc_set_pcm_sink` shipped.
|
||||
Promotes `vc_test_inject_capture` (mono-only, TEST-ONLY) to a public, stereo-capable API.
|
||||
|
||||
@@ -428,7 +428,13 @@ VC_API vc_result vc_set_capture_channels(vc_client* c, uint32_t stream_id, uint3
|
||||
* stream must already be started (vc_stream_start). The core frames, encodes (Opus), seals
|
||||
* (AEAD), and sends (UDP) the provided samples exactly as it would mic/loopback audio.
|
||||
*
|
||||
* samples_per_channel : samples per channel (e.g. 960 for 20 ms @ 48 kHz).
|
||||
* pcm MUST be 48 kHz int16 — the core does NOT resample. (The whole audio engine runs at
|
||||
* 48 kHz; see docs/voice.md §3. A bot is responsible for resampling its source to 48 kHz.)
|
||||
*
|
||||
* samples_per_channel : samples per channel for THIS call. Any count is accepted — the core
|
||||
* buffers and re-chunks to the channel's Opus frame size (the channel's frame_ms decides
|
||||
* this: 480 @ 10 ms, 960 @ 20 ms, 1920 @ 40 ms, …). You need not match the frame size, and
|
||||
* a channel with a non-20 ms window is handled transparently.
|
||||
* channels : 1 (mono) or 2 (stereo interleaved L/R). VC_ERR_INVALID_ARG otherwise.
|
||||
*
|
||||
* Use cases: ReplayKit Broadcast Extension (iOS), ScreenCaptureKit (macOS), bots (TTS /
|
||||
|
||||
@@ -944,7 +944,12 @@ int64_t client_now_ms() {
|
||||
// M2 gap where mode/dtx/complexity/application were silently dropped.
|
||||
voicecat::codec::OpusParams opus_params_from_audio_config(const voicecat::v1::AudioConfig& a) {
|
||||
voicecat::codec::OpusParams p;
|
||||
p.sample_rate = a.sample_rate() ? a.sample_rate() : 48000;
|
||||
// Opus always runs at 48 kHz internally (docs/voice.md §3): the whole AudioEngine clock is
|
||||
// 48 kHz and external PCM is fed at 48 kHz, so the codec must match regardless of what a
|
||||
// channel advertises. Honoring a non-48k effective sample_rate here would create an encoder
|
||||
// expecting e.g. 16k PCM while being fed 48k frames — wrong pitch/duration. The wire
|
||||
// sample_rate field mainly tags narrowband intent; Opus handles that via bitrate at 48k.
|
||||
p.sample_rate = 48000;
|
||||
p.bitrate_bps = a.bitrate_bps() ? a.bitrate_bps() : 24000;
|
||||
p.frame_ms = a.frame_ms() ? a.frame_ms() : 20;
|
||||
p.stereo = (a.mode() == voicecat::v1::MODE_STEREO);
|
||||
@@ -1023,6 +1028,45 @@ void vc_client::on_capture_frame(int kind, const int16_t* pcm, int samples, int
|
||||
// Updated only after the gate above, so a VAD/PTT-closed frame never shows as "talking".
|
||||
ls.last_capture_ms.store(client_now_ms(), std::memory_order_relaxed);
|
||||
|
||||
// The AudioEngine clock is fixed at 48 kHz / 20 ms, so `samples` is always 960. The encoder,
|
||||
// however, wants ls.frame_samples per call, which the channel's frame_ms can make smaller
|
||||
// (480 @10ms) or larger (1920 @40ms). Reframe to that size before encoding (docs/voice.md §3).
|
||||
const int target = ls.frame_samples;
|
||||
if (samples == target) {
|
||||
// Fast path — the common 20 ms channel: encode the engine frame directly, no buffering.
|
||||
encode_and_send_frame(ls, pcm, samples, channels, fd);
|
||||
return;
|
||||
}
|
||||
|
||||
// Reframe: accumulate engine frames and emit target-sized chunks. encode_accum/upmix_scratch
|
||||
// were pre-sized in handle_stream_announce_result, so no allocation happens here.
|
||||
const int ch = std::max(1, channels);
|
||||
if (ls.accum_channels != ch) { // mono/stereo source change — never mix the two
|
||||
ls.accum_count = 0;
|
||||
ls.accum_channels = ch;
|
||||
}
|
||||
const size_t add = static_cast<size_t>(samples) * static_cast<size_t>(ch);
|
||||
const size_t chunk = static_cast<size_t>(target) * static_cast<size_t>(ch);
|
||||
if (ls.accum_count + add > ls.encode_accum.size()) return; // capacity guard (shouldn't trip)
|
||||
std::memcpy(ls.encode_accum.data() + ls.accum_count, pcm, add * sizeof(int16_t));
|
||||
ls.accum_count += add;
|
||||
|
||||
size_t off = 0;
|
||||
while (ls.accum_count - off >= chunk) {
|
||||
encode_and_send_frame(ls, ls.encode_accum.data() + off, target, ch, fd);
|
||||
off += chunk;
|
||||
}
|
||||
if (off > 0) { // shift the sub-frame remainder to the front
|
||||
const size_t rem = ls.accum_count - off;
|
||||
if (rem > 0)
|
||||
std::memmove(ls.encode_accum.data(), ls.encode_accum.data() + off,
|
||||
rem * sizeof(int16_t));
|
||||
ls.accum_count = rem;
|
||||
}
|
||||
}
|
||||
|
||||
void vc_client::encode_and_send_frame(LocalStream& ls, const int16_t* pcm, int samples,
|
||||
int channels, int fd) {
|
||||
uint8_t opus_buf[1500];
|
||||
int opus_len;
|
||||
if (channels == 2) {
|
||||
@@ -1033,13 +1077,13 @@ void vc_client::on_capture_frame(int kind, const int16_t* pcm, int samples, int
|
||||
// Mono capture (mic, or loopback on a mono channel, or test injection) on a channel
|
||||
// configured for stereo — upmix L=R so the stream is still a spec-correct stereo Opus
|
||||
// bitstream. (Mic stays mono in v1 — no stereo capture device — but a stereo channel
|
||||
// requires a stereo bitstream, hence the upmix.)
|
||||
std::vector<int16_t> stereo_pcm(static_cast<size_t>(samples) * 2);
|
||||
// requires a stereo bitstream, hence the upmix.) upmix_scratch is pre-sized at announce.
|
||||
int16_t* st = ls.upmix_scratch.data();
|
||||
for (int i = 0; i < samples; ++i) {
|
||||
stereo_pcm[i * 2] = pcm[i];
|
||||
stereo_pcm[i * 2 + 1] = pcm[i];
|
||||
st[i * 2] = pcm[i];
|
||||
st[i * 2 + 1] = pcm[i];
|
||||
}
|
||||
opus_len = ls.encoder.encode(stereo_pcm.data(), samples, opus_buf, sizeof(opus_buf));
|
||||
opus_len = ls.encoder.encode(st, samples, opus_buf, sizeof(opus_buf));
|
||||
} else {
|
||||
opus_len = ls.encoder.encode(pcm, samples, opus_buf, sizeof(opus_buf));
|
||||
}
|
||||
@@ -1231,6 +1275,18 @@ void vc_client::handle_stream_announce_result(uint64_t req_id,
|
||||
return;
|
||||
}
|
||||
|
||||
// Pre-size the reframe buffers used by on_capture_frame when the channel's frame_ms
|
||||
// differs from the engine's 20 ms (frame_samples != 960). Worst case the accumulator
|
||||
// holds one sub-frame remainder (< frame_samples) plus one engine block (960 samples),
|
||||
// interleaved over up to 2 channels — so (frame_samples + 960) * 2. upmix_scratch sizes
|
||||
// one frame_samples chunk as stereo. Allocated here (control thread), never on the RT
|
||||
// audio thread (architecture.md §3).
|
||||
const size_t fs = ls.frame_samples;
|
||||
ls.encode_accum.assign((fs + 960) * 2, 0);
|
||||
ls.upmix_scratch.assign(fs * 2, 0);
|
||||
ls.accum_count = 0;
|
||||
ls.accum_channels = 0;
|
||||
|
||||
ls.active.store(true, std::memory_order_release);
|
||||
self_uid = self_user_id_;
|
||||
emit_stream_id = ls.stream_id;
|
||||
|
||||
@@ -255,6 +255,19 @@ struct vc_client {
|
||||
// core's WASAPI loopback device. Set from vc_stream_desc::external_feed at stream_start
|
||||
// time and checked in handle_stream_announce_result / stream_stop.
|
||||
bool external_feed = false;
|
||||
|
||||
// Reframe buffer: the AudioEngine clock is fixed at 48 kHz / 20 ms, so capture/feed
|
||||
// always delivers 960-sample frames — but the channel's frame_ms (docs/voice.md §3)
|
||||
// can be 2.5…60 ms, so the encoder needs frame_samples per call (480 @10ms, 1920 @40ms,
|
||||
// …). on_capture_frame accumulates the engine's 960-sample frames here and emits
|
||||
// frame_samples-sized chunks. The 20 ms case (frame_samples == 960) bypasses this
|
||||
// entirely (fast path). Pre-sized at announce; never resized on the audio thread
|
||||
// (architecture.md §3 — no RT-thread allocation). encode_accum holds interleaved int16
|
||||
// at accum_channels; upmix_scratch is the pre-sized mono→stereo upmix target.
|
||||
std::vector<int16_t> encode_accum;
|
||||
size_t accum_count = 0; // flat samples currently buffered
|
||||
int accum_channels = 0; // channel count of buffered data; resets on change
|
||||
std::vector<int16_t> upmix_scratch;
|
||||
};
|
||||
mutable std::mutex local_streams_mu_;
|
||||
std::unordered_map<int, LocalStream> local_streams_; // keyed by vc_stream_kind
|
||||
@@ -337,6 +350,12 @@ struct vc_client {
|
||||
// capture_cb passed to audio_engine_.start(): encode + seal + send one frame for the
|
||||
// given local stream `kind` (M3: multiple concurrent local streams are possible).
|
||||
void on_capture_frame(int kind, const int16_t* pcm, int samples, int channels);
|
||||
// Encode one frame of exactly ls.frame_samples samples-per-channel (upmixing mono→stereo
|
||||
// for a stereo channel as needed), seal it, and send it over UDP, advancing ls.timestamp.
|
||||
// Called by on_capture_frame for each frame_samples-sized chunk. Assumes local_streams_mu_
|
||||
// is held and the send gate/crypto checks have already passed.
|
||||
void encode_and_send_frame(LocalStream& ls, const int16_t* pcm, int samples, int channels,
|
||||
int fd);
|
||||
// Inspect a User proto's streams and wire up any new remote ssrc into audio_engine_,
|
||||
// emitting VC_EVENT_STREAM_STARTED/STOPPED as streams appear/disappear.
|
||||
void sync_remote_streams(const voicecat::v1::User& user);
|
||||
|
||||
@@ -106,6 +106,11 @@ Guidance baked into defaults / docs:
|
||||
- **Frame size: 20 ms default.** Smaller (10 ms) lowers latency at the cost of more
|
||||
per-packet overhead and CPU; larger (40/60 ms) improves efficiency and loss resilience at
|
||||
the cost of latency. Expose it per channel for "low-latency talk" vs "stable music" rooms.
|
||||
The capture engine runs on a fixed 48 kHz / 20 ms clock (960-sample frames), so the send
|
||||
path **reframes** each captured/fed block to the channel's `frame_ms` before encoding
|
||||
(accumulating two 960-frames for a 40 ms channel, splitting each into two 480-frames for a
|
||||
10 ms channel, etc.). This keeps the hardware/`vc_stream_feed_pcm` contract a single 48 kHz
|
||||
clock regardless of the channel's window — see `vc_client::on_capture_frame`.
|
||||
- **Mode/bitrate:** speech channels → `MONO`, `VOIP`, 24–32 kbps, DTX on, FEC on.
|
||||
Music/screen-audio channels → `STEREO`, `AUDIO`, 96–128 kbps, DTX off, FEC optional.
|
||||
- **`application`:** `VOIP` for talk, `AUDIO` for music/screen-share, `LOWDELAY` for
|
||||
|
||||
@@ -202,4 +202,13 @@ if(VOICECAT_USE_VCPKG_DEPS)
|
||||
target_include_directories(test_external_pcm PRIVATE ${VC_TEST_INTERNAL_INCLUDES})
|
||||
add_test(NAME external_pcm COMMAND test_external_pcm)
|
||||
set_tests_properties(external_pcm PROPERTIES TIMEOUT 90)
|
||||
|
||||
# Non-20ms channel frame_ms: the fixed 48k/20ms engine clock is reframed to the channel's
|
||||
# Opus window before encoding. 40ms (accumulate) and 10ms (split) feed->sink round trips.
|
||||
add_executable(test_frame_ms_reframe test_frame_ms_reframe.cpp)
|
||||
target_link_libraries(test_frame_ms_reframe PRIVATE voicecat::server)
|
||||
target_compile_features(test_frame_ms_reframe PRIVATE cxx_std_20)
|
||||
target_include_directories(test_frame_ms_reframe PRIVATE ${VC_TEST_INTERNAL_INCLUDES})
|
||||
add_test(NAME frame_ms_reframe COMMAND test_frame_ms_reframe)
|
||||
set_tests_properties(frame_ms_reframe PROPERTIES TIMEOUT 90)
|
||||
endif()
|
||||
|
||||
325
tests/test_frame_ms_reframe.cpp
Normal file
325
tests/test_frame_ms_reframe.cpp
Normal file
@@ -0,0 +1,325 @@
|
||||
/*
|
||||
* test_frame_ms_reframe — channels whose frame_ms differs from the engine's 20 ms.
|
||||
*
|
||||
* The AudioEngine capture clock is fixed at 48 kHz / 20 ms, so capture and vc_stream_feed_pcm
|
||||
* always deliver 960-sample frames. A channel, however, may set any Opus frame_ms (docs/voice.md
|
||||
* §3: 2.5…60 ms). Before the reframe fix, on_capture_frame handed the engine's 960-sample frame
|
||||
* straight to an encoder configured for the channel's frame_ms: for frame_ms < 20 the receiver
|
||||
* sized its decode buffer too small and dropped every frame (dead audio); for frame_ms > 20 the
|
||||
* channel's setting was silently ignored. on_capture_frame now reframes to ls.frame_samples.
|
||||
*
|
||||
* Two end-to-end cases (admin creates the channel, two guests do a feed→encode→relay→decode→sink
|
||||
* round trip in it). Both assert the sink receives non-zero decoded energy at 48 kHz:
|
||||
*
|
||||
* frame_ms = 40 — larger window: two 960-sample engine frames accumulate into one 1920 encode.
|
||||
* frame_ms = 10 — smaller window: each 960-sample engine frame splits into two 480 encodes.
|
||||
* This is the case that was fully broken (OPUS_BUFFER_TOO_SMALL on decode).
|
||||
*/
|
||||
#include <cstdio>
|
||||
|
||||
#ifdef VOICECAT_HAS_NET
|
||||
|
||||
#include <atomic>
|
||||
#include <chrono>
|
||||
#include <cmath>
|
||||
#include <condition_variable>
|
||||
#include <cstring>
|
||||
#include <filesystem>
|
||||
#include <mutex>
|
||||
#include <string>
|
||||
#include <thread>
|
||||
#include <vector>
|
||||
|
||||
#include "voicecat.h"
|
||||
#include "server.h"
|
||||
#include "db.h"
|
||||
|
||||
// ── Helpers ───────────────────────────────────────────────────────────────────
|
||||
|
||||
static int g_failures = 0;
|
||||
#define CHECK(cond) \
|
||||
do { if (!(cond)) { \
|
||||
std::printf("FAIL [%s:%d]: %s\n", __FILE__, __LINE__, #cond); \
|
||||
++g_failures; \
|
||||
}} while (0)
|
||||
|
||||
struct EventStore {
|
||||
std::mutex mu;
|
||||
std::condition_variable cv;
|
||||
bool auth_ok{false};
|
||||
uint32_t self_user_id{0};
|
||||
bool channel_list_received{false};
|
||||
bool generic_result_received{false};
|
||||
bool generic_ok{false};
|
||||
std::vector<std::pair<uint32_t, uint32_t>> streams_started; // (user_id, stream_id)
|
||||
vc_client* client{nullptr};
|
||||
const char* label{nullptr};
|
||||
};
|
||||
|
||||
static void on_event(void* user, const vc_event* ev) {
|
||||
auto* s = static_cast<EventStore*>(user);
|
||||
std::lock_guard lk(s->mu);
|
||||
switch (ev->type) {
|
||||
case VC_EVENT_SERVER_IDENTITY: vc_confirm_server_identity(s->client, 1); break;
|
||||
case VC_EVENT_AUTH_RESULT:
|
||||
s->auth_ok = (ev->result == VC_OK);
|
||||
s->self_user_id = ev->user_id;
|
||||
break;
|
||||
case VC_EVENT_CHANNEL_LIST: s->channel_list_received = true; break;
|
||||
case VC_EVENT_GENERIC_RESULT:
|
||||
s->generic_result_received = true;
|
||||
s->generic_ok = (ev->result == VC_OK);
|
||||
break;
|
||||
case VC_EVENT_STREAM_STARTED: s->streams_started.emplace_back(ev->user_id, ev->stream_id); break;
|
||||
default: break;
|
||||
}
|
||||
s->cv.notify_all();
|
||||
}
|
||||
|
||||
template <typename Pred>
|
||||
static bool wait_for(EventStore& s, Pred pred, int timeout_ms) {
|
||||
auto deadline = std::chrono::steady_clock::now() + std::chrono::milliseconds(timeout_ms);
|
||||
std::unique_lock lk(s.mu);
|
||||
return s.cv.wait_until(lk, deadline, [&] { return pred(s); });
|
||||
}
|
||||
|
||||
static bool connect_guest(vc_client*& client, const char* name, const char* label,
|
||||
uint16_t port, EventStore& ev) {
|
||||
vc_callbacks cb{on_event, nullptr, &ev};
|
||||
vc_config cfg{label, "0.1", VC_LOG_OFF};
|
||||
client = vc_client_create(&cfg, cb);
|
||||
if (!client) return false;
|
||||
ev.client = client;
|
||||
ev.label = label;
|
||||
if (vc_connect(client, "127.0.0.1", port) != VC_OK) return false;
|
||||
if (vc_authenticate_guest(client, name) != VC_OK) return false;
|
||||
if (!wait_for(ev, [](EventStore& s) { return s.auth_ok; }, 8000)) return false;
|
||||
if (!wait_for(ev, [](EventStore& s) { return s.channel_list_received; }, 3000)) return false;
|
||||
return true;
|
||||
}
|
||||
|
||||
static std::vector<int16_t> make_sine_mono(int n) {
|
||||
std::vector<int16_t> pcm(static_cast<size_t>(n));
|
||||
for (int i = 0; i < n; ++i) {
|
||||
float t = static_cast<float>(i) / 48000.0f;
|
||||
pcm[i] = static_cast<int16_t>(std::sin(2.0f * 3.14159265f * 440.0f * t) * 16000.0f);
|
||||
}
|
||||
return pcm;
|
||||
}
|
||||
|
||||
struct SinkData {
|
||||
std::mutex mu;
|
||||
std::condition_variable cv;
|
||||
std::atomic<int> call_count{0};
|
||||
uint32_t last_sample_rate = 0;
|
||||
int64_t total_energy = 0;
|
||||
};
|
||||
|
||||
static void pcm_sink(void* user, uint32_t, uint32_t,
|
||||
const int16_t* pcm, size_t n, uint32_t channels, uint32_t sr) {
|
||||
auto* d = static_cast<SinkData*>(user);
|
||||
std::lock_guard lk(d->mu);
|
||||
d->last_sample_rate = sr;
|
||||
for (size_t i = 0; i < n * channels; ++i)
|
||||
d->total_energy += std::abs(static_cast<int>(pcm[i]));
|
||||
d->call_count.fetch_add(1, std::memory_order_relaxed);
|
||||
d->cv.notify_all();
|
||||
}
|
||||
|
||||
static bool sink_wait(SinkData& d, int timeout_ms) {
|
||||
auto deadline = std::chrono::steady_clock::now() + std::chrono::milliseconds(timeout_ms);
|
||||
std::unique_lock lk(d.mu);
|
||||
return d.cv.wait_until(lk, deadline, [&] { return d.call_count.load() > 0; });
|
||||
}
|
||||
|
||||
// Admin creates a mono channel with the given frame_ms; returns its id (0 on failure).
|
||||
static uint32_t make_channel(vc_client* admin, EventStore& evAdmin, const char* name,
|
||||
uint32_t frame_ms) {
|
||||
vc_channel_info ch{};
|
||||
ch.name = name;
|
||||
ch.audio.codec = 0; // OPUS
|
||||
ch.audio.mode = 0; // mono
|
||||
ch.audio.sample_rate = 48000;
|
||||
ch.audio.bitrate_bps = 24000;
|
||||
ch.audio.frame_ms = frame_ms;
|
||||
ch.audio.fec = 1;
|
||||
ch.audio.complexity = 5;
|
||||
|
||||
{ std::lock_guard lk(evAdmin.mu); evAdmin.generic_result_received = false; }
|
||||
if (vc_create_channel(admin, &ch) != VC_OK) return 0;
|
||||
if (!wait_for(evAdmin, [](EventStore& s) { return s.generic_result_received; }, 5000)) return 0;
|
||||
{ std::lock_guard lk(evAdmin.mu); if (!evAdmin.generic_ok) return 0; }
|
||||
|
||||
vc_channel_list cl{};
|
||||
if (vc_list_channels(admin, &cl) != VC_OK) return 0;
|
||||
uint32_t id = 0;
|
||||
for (size_t i = 0; i < cl.count; ++i)
|
||||
if (cl.items[i].name && std::string(cl.items[i].name) == name) { id = cl.items[i].id; break; }
|
||||
vc_free_channel_list(&cl);
|
||||
return id;
|
||||
}
|
||||
|
||||
// Full feed→encode→relay→decode→sink round trip inside `channel_id`, asserting the channel's
|
||||
// frame_ms is in effect and the sink hears decoded energy.
|
||||
static void run_case(uint16_t port, vc_client* admin, EventStore& evAdmin,
|
||||
uint32_t channel_id, uint32_t expect_frame_ms, const char* tag) {
|
||||
std::printf("test_frame_ms_reframe[%s]: channel=%u frame_ms=%u\n", tag, channel_id,
|
||||
expect_frame_ms);
|
||||
|
||||
EventStore evA, evB;
|
||||
vc_client *clientA = nullptr, *clientB = nullptr;
|
||||
CHECK(connect_guest(clientA, "ReframeA", "rf-a", port, evA));
|
||||
CHECK(connect_guest(clientB, "ReframeB", "rf-b", port, evB));
|
||||
if (!clientA || !clientB) goto cleanup;
|
||||
|
||||
{
|
||||
uint32_t a_uid = 0, b_uid = 0;
|
||||
{ std::lock_guard lk(evA.mu); a_uid = evA.self_user_id; }
|
||||
{ std::lock_guard lk(evB.mu); b_uid = evB.self_user_id; }
|
||||
|
||||
// Move both guests into the target channel so the relay is channel-scoped to them.
|
||||
CHECK(vc_move_user(admin, a_uid, channel_id) == VC_OK);
|
||||
CHECK(vc_move_user(admin, b_uid, channel_id) == VC_OK);
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(500));
|
||||
|
||||
SinkData sink;
|
||||
CHECK(vc_set_pcm_sink(clientB, pcm_sink, &sink) == VC_OK);
|
||||
|
||||
vc_stream_desc desc{};
|
||||
desc.kind = VC_STREAM_MIC;
|
||||
uint32_t a_sid = 0;
|
||||
CHECK(vc_stream_start(clientA, &desc, &a_sid) == VC_OK);
|
||||
|
||||
bool b_saw_a = wait_for(evB, [&](EventStore& s) {
|
||||
for (auto& [uid, sid] : s.streams_started)
|
||||
if (uid == a_uid) return true;
|
||||
return false;
|
||||
}, 5000);
|
||||
CHECK(b_saw_a);
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(400));
|
||||
|
||||
// The channel's frame_ms must be the effective window on A's stream — proves the encoder
|
||||
// (and hence the reframe target) is configured for the non-20ms window.
|
||||
vc_audio_config ac{};
|
||||
CHECK(vc_get_stream_audio_config(clientA, a_uid, a_sid, &ac) == VC_OK);
|
||||
CHECK(ac.frame_ms == expect_frame_ms);
|
||||
CHECK(ac.sample_rate == 48000);
|
||||
|
||||
// Feed 300 engine-shaped (960-sample / 20 ms / 48 kHz) frames. on_capture_frame reframes
|
||||
// them to the channel's window before encoding.
|
||||
auto sine = make_sine_mono(960);
|
||||
for (int i = 0; i < 300; ++i)
|
||||
CHECK(vc_stream_feed_pcm(clientA, a_sid, sine.data(), 960, 1) == VC_OK);
|
||||
|
||||
CHECK(sink_wait(sink, 5000));
|
||||
std::this_thread::sleep_for(std::chrono::milliseconds(1500));
|
||||
|
||||
int calls = sink.call_count.load();
|
||||
int64_t energy = sink.total_energy;
|
||||
std::printf("test_frame_ms_reframe[%s]: sink calls=%d energy=%lld sr=%u\n",
|
||||
tag, calls, static_cast<long long>(energy), sink.last_sample_rate);
|
||||
CHECK(calls > 0);
|
||||
CHECK(energy > 0); // decoded audio actually arrived
|
||||
CHECK(sink.last_sample_rate == 48000);
|
||||
|
||||
vc_stream_stop(clientA, a_sid);
|
||||
}
|
||||
|
||||
cleanup:
|
||||
if (clientA) { vc_disconnect(clientA); vc_client_destroy(clientA); }
|
||||
if (clientB) { vc_disconnect(clientB); vc_client_destroy(clientB); }
|
||||
std::printf("test_frame_ms_reframe[%s]: done\n", tag);
|
||||
}
|
||||
|
||||
int main() {
|
||||
auto tmp = std::filesystem::temp_directory_path() /
|
||||
("vctest_reframe_" + std::to_string(
|
||||
std::chrono::steady_clock::now().time_since_epoch().count()));
|
||||
std::filesystem::create_directories(tmp);
|
||||
std::string data_dir = tmp.string();
|
||||
|
||||
// Pre-provision an admin so we can create channels with custom frame_ms.
|
||||
{
|
||||
voicecat::server::Database db(data_dir + "/voicecat.db");
|
||||
std::string err;
|
||||
if (!db.open(err) || !db.create_account("admin", "pass", true, err)) {
|
||||
std::printf("FAIL: provision admin: %s\n", err.c_str());
|
||||
std::filesystem::remove_all(tmp);
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
|
||||
std::atomic<uint16_t> bound_port{0};
|
||||
std::mutex ready_mu;
|
||||
std::condition_variable ready_cv;
|
||||
bool ready{false};
|
||||
|
||||
voicecat::server::Config cfg;
|
||||
cfg.data_dir = data_dir;
|
||||
cfg.bind_port = 0;
|
||||
cfg.media_port = 0;
|
||||
cfg.server_name = "VoiceCat-ReframeTest";
|
||||
cfg.allow_guests = true;
|
||||
cfg.on_ready = [&](uint16_t p) {
|
||||
bound_port.store(p);
|
||||
{ std::lock_guard lk(ready_mu); ready = true; }
|
||||
ready_cv.notify_all();
|
||||
};
|
||||
|
||||
voicecat::server::Server server(cfg);
|
||||
std::thread server_thread([&] { server.run(); });
|
||||
{
|
||||
std::unique_lock lk(ready_mu);
|
||||
if (!ready_cv.wait_for(lk, std::chrono::seconds(10), [&] { return ready; })) {
|
||||
std::printf("FAIL: server did not start\n");
|
||||
server.stop(); server_thread.join();
|
||||
std::filesystem::remove_all(tmp);
|
||||
return 1;
|
||||
}
|
||||
}
|
||||
uint16_t port = bound_port.load();
|
||||
std::printf("frame_ms_reframe: server ready on :%u\n", port);
|
||||
|
||||
// Admin client creates the two test channels.
|
||||
EventStore evAdmin;
|
||||
evAdmin.label = "admin";
|
||||
vc_callbacks cbAdmin{on_event, nullptr, &evAdmin};
|
||||
vc_config cfgAdmin{"reframe-admin", "0.1", VC_LOG_OFF};
|
||||
vc_client* admin = vc_client_create(&cfgAdmin, cbAdmin);
|
||||
CHECK(admin != nullptr);
|
||||
evAdmin.client = admin;
|
||||
CHECK(vc_connect(admin, "127.0.0.1", port) == VC_OK);
|
||||
CHECK(vc_authenticate_user(admin, "admin", "pass") == VC_OK);
|
||||
CHECK(wait_for(evAdmin, [](EventStore& s) { return s.auth_ok; }, 8000));
|
||||
CHECK(wait_for(evAdmin, [](EventStore& s) { return s.channel_list_received; }, 3000));
|
||||
|
||||
uint32_t ch40 = make_channel(admin, evAdmin, "Reframe40", 40);
|
||||
uint32_t ch10 = make_channel(admin, evAdmin, "Reframe10", 10);
|
||||
CHECK(ch40 != 0);
|
||||
CHECK(ch10 != 0);
|
||||
|
||||
if (ch40) run_case(port, admin, evAdmin, ch40, 40, "40ms");
|
||||
if (ch10) run_case(port, admin, evAdmin, ch10, 10, "10ms");
|
||||
|
||||
vc_disconnect(admin);
|
||||
vc_client_destroy(admin);
|
||||
server.stop();
|
||||
server_thread.join();
|
||||
std::filesystem::remove_all(tmp);
|
||||
|
||||
if (g_failures == 0) {
|
||||
std::printf("frame_ms_reframe: all checks passed\n");
|
||||
return 0;
|
||||
}
|
||||
std::printf("frame_ms_reframe: %d failure(s)\n", g_failures);
|
||||
return 1;
|
||||
}
|
||||
|
||||
#else // !VOICECAT_HAS_NET
|
||||
|
||||
int main() {
|
||||
std::printf("frame_ms_reframe: SKIP (VOICECAT_HAS_NET not defined)\n");
|
||||
return 0;
|
||||
}
|
||||
|
||||
#endif // VOICECAT_HAS_NET
|
||||
Reference in New Issue
Block a user