diff --git a/CLAUDE.md b/CLAUDE.md index e95c6ab..aeb760a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -8,8 +8,9 @@ and what's next* read [`PROGRESS.md`](PROGRESS.md); for *design* read [`docs/`]( > server-mute, channel CRUD, in-app account management, disconnect/keepalive/reaper. Windows > WinForms C# client shipped (M4). **macOS AppKit client shipped** — `VoiceCatMac.xcodeproj` > at `clients/apple/macOS/`. **iOS SwiftUI client shipped** — `VoiceCatiOS.xcodeproj` at -> `clients/apple/iOS/`. `ctest --preset dev` green — 21/21 tests. -> Next: ReplayKit Broadcast Extension or DRED/audio polish. See [`PROGRESS.md`](PROGRESS.md). +> `clients/apple/iOS/`. `ctest --preset dev` green — 23/23 tests. +> External PCM feed/tap API (`vc_stream_feed_pcm` + `vc_set_pcm_sink`) shipped. +> Next: ReplayKit Broadcast Extension (iOS) or ScreenCaptureKit (macOS). See [`PROGRESS.md`](PROGRESS.md). VoiceCat = self-hosted native voice & text chat (TeamSpeak/Mumble-style). Plain TCP (control) + UDP (media), no WebRTC, encrypted by default. A shared C++ core (`libvoicecat`) drives diff --git a/PROGRESS.md b/PROGRESS.md index ac1ac9e..9d2c02a 100644 --- a/PROGRESS.md +++ b/PROGRESS.md @@ -39,12 +39,12 @@ up instantly. Newest status at the top. `allowBluetoothA2DP` and the output route must remain the headphones/A2DP device — NOT flip to `…Record`. If confirmed, delete the `logSessionState` calls + method and the prior band-aid comments in `IOSAudioRouter`/`audio_engine.cpp` can be trimmed. - - **Verified on Windows:** `cmake --build --preset dev` clean, `ctest --preset dev` 22/22 - (21/21 prior + new `test_dred_toggle`). - iOS build & on-device run still to be done by the user on the Mac. + - **Verified on Windows:** `cmake --build --preset dev` clean, `ctest --preset dev` 23/23 + (22/22 prior + `test_external_pcm` new binary). iOS build & on-device run still to be done by the user on the Mac. -- **Planned (not started):** **External PCM feed/tap API (`vc_stream_feed_pcm` + - `vc_set_pcm_sink`)** (2026-06-19, plan written on Windows; implement on Mac). A public, +- **Done (2026-06-20):** **External PCM feed/tap API (`vc_stream_feed_pcm` + + `vc_set_pcm_sink`)** — see detail in M5 section below. `ctest --preset dev` 23/23 (was 22/22 + 1 new test binary with 3 sub-tests). + Next: iOS ReplayKit and macOS ScreenCaptureKit consumers of this API. A public, documented API for driving audio streams with externally-provided PCM instead of (or in addition to) miniaudio's hardware device. Motivated by four concrete use cases — all in our roadmap — that the current "miniaudio owns the device" model can't serve: @@ -208,7 +208,17 @@ up instantly. Newest status at the top. ## Recent completed work -All items below are `[x]` done; `ctest --preset dev` 21/21 on Windows after each. +All items below are `[x]` done; `ctest --preset dev` 23/23 on Windows after all. + +- **External PCM feed/tap API** (2026-06-20): `vc_stream_feed_pcm` + `vc_set_pcm_sink` shipped. + Promotes `vc_test_inject_capture` (mono-only, TEST-ONLY) to a public, stereo-capable API. + Adds symmetric PCM sink on the playback thread. Swift wrapper (`feedPcm`/`setPcmSink` in + `VoiceCatClient.swift`, 4 XCTest smoke tests). C# wrapper (`StreamFeedPcm`/`SetPcmSink` in + `VoiceCatClient.cs` + `NativeMethods.cs`, 4 xUnit smoke tests in `ExternalPcmTests.cs`). + Three new headless C++ ctests. Docs: architecture.md §4 new subsection, voice.md §9 updated, + protocol.md §8 explicit no-protocol-change note, roadmap.md M5 entry. Files: `voicecat.h`, + `voicecat.cpp`, `client.{h,cpp}`, `audio_engine.{h,cpp}`, `tests/test_external_pcm.cpp`, + `tests/CMakeLists.txt`, Swift + C# wrappers. - **iOS A2DP + stereo root cause fix** (2026-06-20): miniaudio's NULL-context `ma_device_init` was calling `AVAudioSession setCategory(Record)` on every device open, wiping the session @@ -412,8 +422,14 @@ iOS 18.0 deployment target. App Group `group.cat.voice.VoiceCat` for Keychain sh on macOS). Implement via `vc_stream_feed_pcm` + `SCStream` once the feed API ships. - [ ] **iOS ReplayKit Broadcast Extension** (`VoiceCatBroadcast`) — separate Xcode target, App Group credential sharing, `SampleHandler.swift`. Implement via `vc_stream_feed_pcm`. -- [ ] **External PCM feed/tap API** (`vc_stream_feed_pcm` + `vc_set_pcm_sink`) — see full - plan in "Where we left off" above. +- [x] **External PCM feed/tap API** (`vc_stream_feed_pcm` + `vc_set_pcm_sink`) — done + 2026-06-20. Promotes `vc_test_inject_capture` (mono-only, TEST-ONLY) to a public API with + stereo support. Adds a symmetric PCM sink fired on the playback thread per decoded remote + stream. Full wrappers for Swift (`feedPcm`/`setPcmSink`) and C# (`StreamFeedPcm`/ + `SetPcmSink`). Three new C++ ctests (`test_feed_pcm_round_trip`, `test_feed_pcm_stereo`, + `test_pcm_sink`), 4 Swift XCTest smoke tests, 4 C# xUnit smoke tests. Docs updated + (architecture.md §4 new subsection, voice.md §9 updated, protocol.md §8 explicit + no-protocol-change note, roadmap.md M5 entry). `ctest --preset dev` 23/23. --- diff --git a/clients/apple/Sources/VoiceCatCore/VoiceCatClient.swift b/clients/apple/Sources/VoiceCatCore/VoiceCatClient.swift index 59ce3ae..92a62c8 100644 --- a/clients/apple/Sources/VoiceCatCore/VoiceCatClient.swift +++ b/clients/apple/Sources/VoiceCatCore/VoiceCatClient.swift @@ -309,6 +309,46 @@ public final class VoiceCatClient { VoiceCatResult(vc_set_capture_channels(handle, streamId, channels)) } + // MARK: - External PCM feed / tap + + /// External PCM feed — drives a local stream's encode pipeline with caller-supplied PCM + /// instead of (or in addition to) a hardware capture device. Intended for ReplayKit + /// Broadcast Extension (iOS), ScreenCaptureKit (macOS), bots, and soundboard use cases. + /// + /// - Parameters: + /// - streamId: The stream returned by `startStream`. Must be active. + /// - pcm: Raw int16 PCM pointer. Caller must keep the buffer alive for the duration of the call. + /// - samplesPerChannel: Samples per channel (e.g. 960 for 20 ms @ 48 kHz). + /// - channels: 1 (mono) or 2 (stereo interleaved L/R). + @discardableResult + public func feedPcm(streamId: UInt32, pcm: UnsafePointer, + samplesPerChannel: Int, channels: UInt32) -> VoiceCatResult { + VoiceCatResult(vc_stream_feed_pcm(handle, streamId, pcm, + UInt(samplesPerChannel), channels)) + } + + /// Convenience overload for feeding from a Swift `[Int16]` array. + @discardableResult + public func feedPcm(streamId: UInt32, pcm: [Int16], + samplesPerChannel: Int, channels: UInt32) -> VoiceCatResult { + pcm.withUnsafeBufferPointer { + feedPcm(streamId: streamId, pcm: $0.baseAddress!, + samplesPerChannel: samplesPerChannel, channels: channels) + } + } + + /// External PCM tap — receive decoded per-stream audio as raw int16 PCM before it + /// reaches the hardware mix. Fires once per decoded Opus frame per remote stream. + /// + /// The callback is a C function pointer (`@convention(c)`) receiving: + /// `(user, userId, streamId, pcm, samplesPerChannel, channels, sampleRate)` + /// + /// Pass `nil` to disable (default). The callback MUST NOT block or allocate. + @discardableResult + public func setPcmSink(_ cb: vc_pcm_sink_cb?, user: UnsafeMutableRawPointer?) -> VoiceCatResult { + VoiceCatResult(vc_set_pcm_sink(handle, cb, user)) + } + @discardableResult public func setInputMode(_ mode: VoiceCatInputMode) -> VoiceCatResult { VoiceCatResult(vc_set_input_mode(handle, mode.cValue)) diff --git a/clients/apple/Tests/VoiceCatCoreTests/ExternalPcmTests.swift b/clients/apple/Tests/VoiceCatCoreTests/ExternalPcmTests.swift new file mode 100644 index 0000000..c0d648a --- /dev/null +++ b/clients/apple/Tests/VoiceCatCoreTests/ExternalPcmTests.swift @@ -0,0 +1,68 @@ +// ExternalPcmTests — Swift wrapper smoke tests for vc_stream_feed_pcm / vc_set_pcm_sink. +// +// These tests verify that the Swift API surface compiles, is callable, and returns expected +// results at the C-ABI boundary — without requiring a live server or audio hardware. +// Full end-to-end relay / decode verification is covered by tests/test_external_pcm.cpp +// (C++ ctest), which runs headlessly on all platforms. + +import XCTest +@testable import VoiceCatCore + +final class ExternalPcmTests: XCTestCase { + + // MARK: - feedPcm: API surface smoke + + /// Calling feedPcm without a connected client or active stream must return .invalidArg + /// (not crash). Proves the Swift→C bridge compiles and handles the error path. + func testFeedPcm_noActiveStream_returnsInvalidArg() { + let client = VoiceCatClient(config: VoiceCatConfig( + clientName: "ext-pcm-test", + clientVersion: "0.1", + logLevel: .off + )) + let sine = [Int16](repeating: 0, count: 960) + // Stream 0 doesn't exist — the core must return invalidArg, not crash. + let result = client.feedPcm(streamId: 0, pcm: sine, samplesPerChannel: 960, channels: 1) + XCTAssertEqual(result, .invalidArg) + } + + /// Calling feedPcm with channels=3 (invalid) must return .invalidArg. + func testFeedPcm_invalidChannels_returnsInvalidArg() { + let client = VoiceCatClient(config: VoiceCatConfig( + clientName: "ext-pcm-test", + clientVersion: "0.1", + logLevel: .off + )) + let pcm = [Int16](repeating: 0, count: 960 * 3) + let result = client.feedPcm(streamId: 0, pcm: pcm, samplesPerChannel: 960, channels: 3) + XCTAssertEqual(result, .invalidArg) + } + + // MARK: - setPcmSink: API surface smoke + + /// setPcmSink(nil) on a freshly-created client must succeed (nil = disable, which is the + /// default state — a no-op that must still return .ok). + func testSetPcmSink_nil_returnsOk() { + let client = VoiceCatClient(config: VoiceCatConfig( + clientName: "ext-pcm-test", + clientVersion: "0.1", + logLevel: .off + )) + let result = client.setPcmSink(nil, user: nil) + XCTAssertEqual(result, .ok) + } + + /// Calling setPcmSink with a @convention(c) function and then immediately disabling it + /// with nil must both succeed. Verifies the C-ABI function-pointer round-trip. + func testSetPcmSink_enableThenDisable_bothSucceed() { + let client = VoiceCatClient(config: VoiceCatConfig( + clientName: "ext-pcm-test", + clientVersion: "0.1", + logLevel: .off + )) + + let mySink: vc_pcm_sink_cb = { _, _, _, _, _, _, _ in } + XCTAssertEqual(client.setPcmSink(mySink, user: nil), .ok) + XCTAssertEqual(client.setPcmSink(nil, user: nil), .ok) + } +} diff --git a/clients/windows/VoiceCat.Interop.Tests/ExternalPcmTests.cs b/clients/windows/VoiceCat.Interop.Tests/ExternalPcmTests.cs new file mode 100644 index 0000000..6b3610c --- /dev/null +++ b/clients/windows/VoiceCat.Interop.Tests/ExternalPcmTests.cs @@ -0,0 +1,55 @@ +using System.Runtime.InteropServices; +using VoiceCat.Interop; + +namespace VoiceCat.Interop.Tests; + +/// +/// API-surface smoke tests for StreamFeedPcm / SetPcmSink through the P/Invoke layer. +/// These tests need voicecat.dll but NOT a running server — they verify the C# wrapper +/// compiles, is callable, and the error paths work at the C-ABI boundary. Full E2E relay +/// verification is handled by tests/test_external_pcm.cpp (C++ ctest). +/// +public sealed class ExternalPcmTests +{ + [Fact] + public void StreamFeedPcm_NoActiveStream_ReturnsInvalidArg() + { + using var client = new VoiceCatClient("ext-pcm-test", "0.1", VcLogLevel.Off); + var pcm = new short[960]; + // Stream 0 doesn't exist — core must return InvalidArg, not crash. + var result = client.StreamFeedPcm(0, pcm, 960, channels: 1); + Assert.Equal(VcResult.InvalidArg, result); + } + + [Fact] + public void StreamFeedPcm_InvalidChannels_ReturnsInvalidArg() + { + using var client = new VoiceCatClient("ext-pcm-test", "0.1", VcLogLevel.Off); + var pcm = new short[960 * 3]; + // channels=3 is not supported — trampoline rejects before touching audio state. + var result = client.StreamFeedPcm(0, pcm, 960, channels: 3); + Assert.Equal(VcResult.InvalidArg, result); + } + + [Fact] + public void SetPcmSink_Zero_ReturnsOk() + { + using var client = new VoiceCatClient("ext-pcm-test", "0.1", VcLogLevel.Off); + // IntPtr.Zero = disable — default state, must be a no-op that returns Ok. + var result = client.SetPcmSink(IntPtr.Zero, IntPtr.Zero); + Assert.Equal(VcResult.Ok, result); + } + + [Fact] + public void SetPcmSink_EnableThenDisable_BothSucceed() + { + using var client = new VoiceCatClient("ext-pcm-test", "0.1", VcLogLevel.Off); + + // Keep the delegate alive until after we unregister it (not just until the P/Invoke call). + NativeMethods.VcPcmSinkCallback sink = static (_, _, _, _, _, _, _) => { }; + nint fp = Marshal.GetFunctionPointerForDelegate(sink); + Assert.Equal(VcResult.Ok, client.SetPcmSink(fp, IntPtr.Zero)); + Assert.Equal(VcResult.Ok, client.SetPcmSink(IntPtr.Zero, IntPtr.Zero)); + GC.KeepAlive(sink); + } +} diff --git a/clients/windows/VoiceCat.Interop/NativeMethods.cs b/clients/windows/VoiceCat.Interop/NativeMethods.cs index edae603..1f88deb 100644 --- a/clients/windows/VoiceCat.Interop/NativeMethods.cs +++ b/clients/windows/VoiceCat.Interop/NativeMethods.cs @@ -98,6 +98,25 @@ internal static partial class NativeMethods internal static unsafe partial VcResult vc_test_inject_capture(nint c, uint streamId, short* pcm, nuint samples); + // ── External PCM feed / tap ────────────────────────────────────────────────────────── + + [LibraryImport(LibName)] + internal static unsafe partial VcResult vc_stream_feed_pcm(nint c, uint streamId, + short* pcm, nuint samplesPerChannel, uint channels); + + // Delegate type for the PCM sink callback — callers convert to a native function + // pointer via Marshal.GetFunctionPointerForDelegate (for instance members) or by casting + // a static lambda to delegate* unmanaged<> (for [UnmanagedCallersOnly] statics). + // Keep the delegate alive for the lifetime of the sink registration. + [UnmanagedFunctionPointer(CallingConvention.Cdecl)] + public delegate void VcPcmSinkCallback(IntPtr user, uint userId, uint streamId, + IntPtr pcm, nuint samplesPerChannel, uint channels, uint sampleRate); + + // cb is a raw function pointer (IntPtr.Zero = disable). Use + // Marshal.GetFunctionPointerForDelegate(sinkDelegate) to convert from VcPcmSinkCallback. + [LibraryImport(LibName)] + internal static partial VcResult vc_set_pcm_sink(nint c, nint cb, IntPtr user); + // ── Text ───────────────────────────────────────────────────────────────────────────── [LibraryImport(LibName, StringMarshalling = StringMarshalling.Utf8)] internal static partial VcResult vc_send_text(nint c, VcTextScope scope, uint targetId, diff --git a/clients/windows/VoiceCat.Interop/VoiceCatClient.cs b/clients/windows/VoiceCat.Interop/VoiceCatClient.cs index aa6b9f2..418d90d 100644 --- a/clients/windows/VoiceCat.Interop/VoiceCatClient.cs +++ b/clients/windows/VoiceCat.Interop/VoiceCatClient.cs @@ -225,6 +225,20 @@ public sealed class VoiceCatClient : IDisposable return (r, r == VcResult.Ok ? Marshaling.ToManaged(in native) : null); } + // ── External PCM feed / tap ───────────────────────────────────────────────────────── + public unsafe VcResult StreamFeedPcm(uint streamId, ReadOnlySpan pcm, + int samplesPerChannel, uint channels) + { + fixed (short* p = pcm) + return NativeMethods.vc_stream_feed_pcm(_handle.DangerousGetHandle(), + streamId, p, (nuint)samplesPerChannel, channels); + } + + // Pass Marshal.GetFunctionPointerForDelegate(cb) for a managed delegate, or + // IntPtr.Zero to disable. Keep the delegate alive for the lifetime of the registration. + public VcResult SetPcmSink(nint cb, IntPtr user) => + NativeMethods.vc_set_pcm_sink(_handle.DangerousGetHandle(), cb, user); + // ── M5: Moderation & admin ─────────────────────────────────────────────────────────── public VcResult KickUser(uint userId, string? reason = null) => NativeMethods.vc_kick_user(_handle.DangerousGetHandle(), userId, reason); diff --git a/core/include/voicecat.h b/core/include/voicecat.h index 2069fbb..6ea5269 100644 --- a/core/include/voicecat.h +++ b/core/include/voicecat.h @@ -413,6 +413,45 @@ VC_API vc_result vc_test_inject_capture(vc_client* c, uint32_t stream_id, const * VC_ERR_INVALID_ARG if stream_id is unknown or channels is not 1 or 2. */ VC_API vc_result vc_set_capture_channels(vc_client* c, uint32_t stream_id, uint32_t channels); +/* ── External PCM feed/tap ─────────────────────────────────────────────────── */ + +/* External PCM feed — production-grade API for driving a local stream's encode pipeline + * with caller-supplied PCM instead of (or in addition to) a hardware capture device. The + * stream must already be started (vc_stream_start). The core frames, encodes (Opus), seals + * (AEAD), and sends (UDP) the provided samples exactly as it would mic/loopback audio. + * + * samples_per_channel : samples per channel (e.g. 960 for 20 ms @ 48 kHz). + * channels : 1 (mono) or 2 (stereo interleaved L/R). VC_ERR_INVALID_ARG otherwise. + * + * Use cases: ReplayKit Broadcast Extension (iOS), ScreenCaptureKit (macOS), bots (TTS / + * music / relay), soundboards, DAW integration. Works for any stream kind (MIC / + * SCREEN_AUDIO / AUX_DEVICE). Thread-safe; may be called from any thread. + * + * Replaces vc_test_inject_capture (deprecated alias, see below). */ +VC_API vc_result vc_stream_feed_pcm(vc_client* c, uint32_t stream_id, + const int16_t* pcm, size_t samples_per_channel, + uint32_t channels); + +/* External PCM tap — receive decoded remote audio as int16 PCM per stream, before it is + * summed into the hardware mix. The callback fires on the audio playback thread once per + * decoded Opus frame (typically every 20 ms) for each active remote stream: + * + * cb(user, user_id, stream_id, pcm, samples_per_channel, channels, sample_rate) + * + * user_id / stream_id : identify the sender (same values as VC_EVENT_STREAM_STARTED). + * pcm : decoded int16 PCM, interleaved when channels == 2. + * samples_per_channel : samples per channel for this frame (typically 960 @ 48 kHz). + * channels : 1 or 2, matching the sender's stream configuration. + * sample_rate : always 48000 in the current implementation. + * + * Pass cb = NULL to disable (default: disabled; hardware playback only). + * The callback MUST NOT block, lock, or allocate — copy what you need and return. + * PCM is still delivered to the hardware playback device regardless (dual output). */ +typedef void (*vc_pcm_sink_cb)(void* user, uint32_t user_id, uint32_t stream_id, + const int16_t* pcm, size_t samples_per_channel, + uint32_t channels, uint32_t sample_rate); +VC_API vc_result vc_set_pcm_sink(vc_client* c, vc_pcm_sink_cb cb, void* user); + /* ── Text ─────────────────────────────────────────────────────────────────── */ VC_API vc_result vc_send_text(vc_client* c, vc_text_scope scope, uint32_t target_id, const char* utf8); diff --git a/core/src/audio/audio_engine.cpp b/core/src/audio/audio_engine.cpp index 62b1803..6ecd741 100644 --- a/core/src/audio/audio_engine.cpp +++ b/core/src/audio/audio_engine.cpp @@ -359,7 +359,11 @@ bool AudioEngine::resume() { #endif } -void AudioEngine::inject_capture(int kind, const int16_t* pcm, size_t n) { +void AudioEngine::inject_capture(int kind, const int16_t* pcm, size_t samples_per_channel, + int channels) { + const int ch = std::max(1, channels); + const size_t n = samples_per_channel * static_cast(ch); + InjectTap* tap; { std::lock_guard lk(inject_mu_); @@ -371,26 +375,38 @@ void AudioEngine::inject_capture(int kind, const int16_t* pcm, size_t n) { tap = slot.get(); } + // If the channel count changed, reset the ring to avoid mixing mono and stereo samples. + if (tap->channels != ch) { + tap->write.store(0, std::memory_order_relaxed); + tap->read.store(0, std::memory_order_relaxed); + tap->channels = ch; + } + size_t w = tap->write.load(std::memory_order_relaxed); for (size_t i = 0; i < n; ++i) tap->ring[(w + i) % kInjectCapSamples] = pcm[i]; tap->write.store(w + n, std::memory_order_release); - // Fire capture_cb_ for each complete frame now available. + // Fire capture_cb_ for each complete frame (frame_samples_ * ch flat samples). + const size_t frame_flat = static_cast(frame_samples_) * static_cast(ch); while (true) { size_t r = tap->read.load(std::memory_order_relaxed); size_t avail = tap->write.load(std::memory_order_acquire) - r; - if (avail < static_cast(frame_samples_)) break; + if (avail < frame_flat) break; - std::vector frame(frame_samples_); - for (int i = 0; i < frame_samples_; ++i) + std::vector frame(frame_flat); + for (size_t i = 0; i < frame_flat; ++i) frame[i] = tap->ring[(r + i) % kInjectCapSamples]; - tap->read.store(r + frame_samples_, std::memory_order_release); + tap->read.store(r + frame_flat, std::memory_order_release); - if (capture_cb_) capture_cb_(kind, frame.data(), frame_samples_, 1); + if (capture_cb_) capture_cb_(kind, frame.data(), frame_samples_, ch); } } +void AudioEngine::inject_capture(int kind, const int16_t* pcm, size_t n) { + inject_capture(kind, pcm, n, 1); +} + void AudioEngine::push_recv_frame(uint32_t ssrc, JitterBuffer::Frame f) { std::lock_guard lk(streams_mu_); auto& s = streams_[ssrc]; @@ -472,9 +488,12 @@ uint32_t AudioEngine::stream_target_depth_ms(uint32_t ssrc) const { } #ifdef VOICECAT_HAS_OPUS -void AudioEngine::init_recv_stream(uint32_t ssrc, const codec::OpusParams& p) { +void AudioEngine::init_recv_stream(uint32_t ssrc, const codec::OpusParams& p, + uint32_t user_id, uint32_t stream_id) { std::lock_guard lk(streams_mu_); auto& stream = streams_[ssrc]; + stream.user_id = user_id; + stream.stream_id = stream_id; stream.decoder.init(p); // Ring must be sized for this decoder's actual channel/frame-size — see RemoteStream::ring // comment in audio_engine.h for why this can't just be the playback callback's frame count. @@ -491,6 +510,11 @@ void AudioEngine::init_recv_stream(uint32_t ssrc, const codec::OpusParams& p) { } #endif +void AudioEngine::set_pcm_sink(PcmSink cb, void* user) { + pcm_sink_user_.store(user, std::memory_order_relaxed); + pcm_sink_.store(cb, std::memory_order_release); +} + #ifdef VOICECAT_HAS_AUDIO void AudioEngine::capture_data_cb(ma_device* dev, void* /*out*/, @@ -634,6 +658,19 @@ void AudioEngine::on_playback(int16_t* out, ma_uint32 frames) { stream.recv_ns->process_capture(stream.decode_scratch.data(), n, static_cast(params_.sample_rate)); + // PCM sink: deliver decoded per-stream audio to external consumer (bots, + // transcription, recording) before it enters the hardware mix. Atomic relaxed- + // load is safe on the RT thread — the fn-ptr and user-ptr are independent + // pointer-sized values written together by set_pcm_sink (release store). + if (auto sink = pcm_sink_.load(std::memory_order_relaxed)) { + sink(pcm_sink_user_.load(std::memory_order_relaxed), + stream.user_id, stream.stream_id, + stream.decode_scratch.data(), + static_cast(n), + static_cast(dec_channels), + params_.sample_rate); + } + stream.push_ring(stream.decode_scratch.data(), static_cast(n)); stream.playout_ts += static_cast(n); } diff --git a/core/src/audio/audio_engine.h b/core/src/audio/audio_engine.h index ba4ee83..060e6b1 100644 --- a/core/src/audio/audio_engine.h +++ b/core/src/audio/audio_engine.h @@ -190,10 +190,22 @@ class AudioEngine { #ifdef VOICECAT_HAS_OPUS // Configure the Opus decoder for an incoming ssrc (must be called before - // push_recv_frame for that ssrc). Thread-safe. - void init_recv_stream(uint32_t ssrc, const codec::OpusParams& p); + // push_recv_frame for that ssrc). user_id/stream_id identify the source for the + // pcm_sink_ callback. Thread-safe. + void init_recv_stream(uint32_t ssrc, const codec::OpusParams& p, + uint32_t user_id, uint32_t stream_id); #endif + // External PCM tap: callback fired once per decoded Opus frame per remote stream, on the + // playback (RT) thread. Matching signature to vc_pcm_sink_cb (cast at the C-ABI boundary). + // Pass nullptr to disable. Thread-safe (atomic store; the RT read is relaxed-load). + using PcmSink = void(*)(void*, uint32_t, uint32_t, const int16_t*, size_t, uint32_t, uint32_t); + void set_pcm_sink(PcmSink cb, void* user); + + // External PCM feed overload: stereo-aware variant of inject_capture. samples_per_channel + // is samples per channel; total samples written = samples_per_channel * channels. + void inject_capture(int kind, const int16_t* pcm, size_t samples_per_channel, int channels); + #ifdef VOICECAT_HAS_AUDIO // TEST-ONLY — exposes the playback mixer without a real ma_device, so tests can verify // stereo mixing end-to-end (no audio hardware needed). Same logic the real playback @@ -364,6 +376,7 @@ class AudioEngine { std::vector ring; // circular, size = kInjectCapSamples std::atomic write{0}; std::atomic read{0}; + int channels{1}; // channel count last written; resets ring on change }; std::mutex inject_mu_; std::unordered_map> inject_taps_; @@ -405,6 +418,11 @@ class AudioEngine { #endif std::vector dred_payload_scratch_; // pre-sized to 4000 bytes + // Source identity: stored at init_recv_stream() so the pcm_sink_ callback can receive + // (user_id, stream_id) without a separate map lookup from the RT playback thread. + uint32_t user_id = 0; + uint32_t stream_id = 0; + // M3: talk-indicator edge detection (docs/voice.md §7) — updated by push_recv_frame // (already off the real-time audio thread), polled by poll_talk_transitions(). std::atomic last_voice_ms{0}; @@ -468,6 +486,11 @@ class AudioEngine { int frame_samples_ = 960; // 20 ms @48 kHz + // External PCM tap: atomic fn-ptr + user-ptr pair. Written by set_pcm_sink (any thread); + // read by on_playback (RT thread) via relaxed load — safe for pointer-sized atomics. + std::atomic pcm_sink_{nullptr}; + std::atomic pcm_sink_user_{nullptr}; + #ifdef VOICECAT_HAS_OPUS ::OpusDREDDecoder* dred_dec_ = nullptr; // shared DRED decoder; null if unsupported #endif diff --git a/core/src/core/client.cpp b/core/src/core/client.cpp index e9f356b..60869b4 100644 --- a/core/src/core/client.cpp +++ b/core/src/core/client.cpp @@ -1109,7 +1109,7 @@ void vc_client::sync_remote_streams(const voicecat::v1::User& user) { remote_streams_[ssrc] = {user.id(), si.stream_id()}; voicecat::codec::OpusParams p = opus_params_from_audio_config(si.audio()); - audio_engine_.init_recv_stream(ssrc, p); + audio_engine_.init_recv_stream(ssrc, p, user.id(), si.stream_id()); bool muted = self_deafened_.load(std::memory_order_acquire) || server_deafened_.load(std::memory_order_acquire); audio_engine_.set_stream_mute(ssrc, muted); @@ -1490,7 +1490,8 @@ vc_result vc_client::get_stream_audio_config(uint32_t user_id, uint32_t stream_i return VC_ERR_INVALID_ARG; } -vc_result vc_client::test_inject_capture(uint32_t stream_id, const int16_t* pcm, size_t samples) { +vc_result vc_client::stream_feed_pcm(uint32_t stream_id, const int16_t* pcm, + size_t samples_per_channel, uint32_t channels) { int kind = -1; { std::lock_guard lk(local_streams_mu_); @@ -1502,10 +1503,20 @@ vc_result vc_client::test_inject_capture(uint32_t stream_id, const int16_t* pcm, } } if (kind < 0) return VC_ERR_INVALID_ARG; - audio_engine_.inject_capture(kind, pcm, samples); + audio_engine_.inject_capture(kind, pcm, samples_per_channel, static_cast(channels)); return VC_OK; } +vc_result vc_client::set_pcm_sink(vc_pcm_sink_cb cb, void* user) { + audio_engine_.set_pcm_sink( + reinterpret_cast(cb), user); + return VC_OK; +} + +vc_result vc_client::test_inject_capture(uint32_t stream_id, const int16_t* pcm, size_t samples) { + return stream_feed_pcm(stream_id, pcm, samples, 1); +} + vc_result vc_client::list_devices(vc_device_kind kind, vc_device_list* out) { #ifdef VOICECAT_HAS_AUDIO // Works in any connection state — device pickers need to populate pre-connect. @@ -1876,6 +1887,12 @@ vc_result vc_client::list_devices(vc_device_kind, vc_device_list* out) { vc_result vc_client::get_stream_audio_config(uint32_t, uint32_t, vc_audio_config*) { return VC_ERR_NOT_IMPLEMENTED; } +vc_result vc_client::stream_feed_pcm(uint32_t, const int16_t*, size_t, uint32_t) { + return VC_ERR_NOT_IMPLEMENTED; +} +vc_result vc_client::set_pcm_sink(vc_pcm_sink_cb, void*) { + return VC_ERR_NOT_IMPLEMENTED; +} vc_result vc_client::test_inject_capture(uint32_t, const int16_t*, size_t) { return VC_ERR_NOT_IMPLEMENTED; } diff --git a/core/src/core/client.h b/core/src/core/client.h index 7b90d38..08b6bd8 100644 --- a/core/src/core/client.h +++ b/core/src/core/client.h @@ -81,7 +81,15 @@ struct vc_client { vc_result get_stream_audio_config(uint32_t user_id, uint32_t stream_id, vc_audio_config* out); - // TEST-ONLY (see voicecat.h) — inject synthetic PCM into a local stream's encode pipeline. + // External PCM feed (see voicecat.h: vc_stream_feed_pcm). Production API for driving a + // local stream's encode pipeline without a hardware capture device. channels = 1 or 2. + vc_result stream_feed_pcm(uint32_t stream_id, const int16_t* pcm, + size_t samples_per_channel, uint32_t channels); + + // External PCM sink (see voicecat.h: vc_set_pcm_sink). Delegates to AudioEngine. + vc_result set_pcm_sink(vc_pcm_sink_cb cb, void* user); + + // TEST-ONLY (see voicecat.h) — deprecated alias for stream_feed_pcm(..., channels=1). vc_result test_inject_capture(uint32_t stream_id, const int16_t* pcm, size_t samples); // M5: moderation & admin. diff --git a/core/src/voicecat.cpp b/core/src/voicecat.cpp index e2f66f6..bdcf861 100644 --- a/core/src/voicecat.cpp +++ b/core/src/voicecat.cpp @@ -143,7 +143,19 @@ vc_result vc_get_stream_audio_config(vc_client* c, uint32_t user_id, uint32_t st vc_result vc_test_inject_capture(vc_client* c, uint32_t stream_id, const int16_t* pcm, size_t samples) { if (c == nullptr || pcm == nullptr) return VC_ERR_INVALID_ARG; - return c->test_inject_capture(stream_id, pcm, samples); + return c->stream_feed_pcm(stream_id, pcm, samples, 1); +} + +vc_result vc_stream_feed_pcm(vc_client* c, uint32_t stream_id, const int16_t* pcm, + size_t samples_per_channel, uint32_t channels) { + if (c == nullptr || pcm == nullptr) return VC_ERR_INVALID_ARG; + if (channels != 1 && channels != 2) return VC_ERR_INVALID_ARG; + return c->stream_feed_pcm(stream_id, pcm, samples_per_channel, channels); +} + +vc_result vc_set_pcm_sink(vc_client* c, vc_pcm_sink_cb cb, void* user) { + if (c == nullptr) return VC_ERR_INVALID_ARG; + return c->set_pcm_sink(cb, user); } vc_result vc_set_capture_channels(vc_client* c, uint32_t stream_id, uint32_t channels) { diff --git a/docs/architecture.md b/docs/architecture.md index 5c57e20..acc4842 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -155,6 +155,26 @@ Design notes: `PostMessage` with no meaningful latency cost. `VoiceCatClientHandle : SafeHandle` wraps the `vc_client*` and guarantees `vc_client_destroy` runs on GC/Dispose. +### External PCM feed/tap + +Two API functions let callers bypass miniaudio entirely for a stream: + +| Function | Direction | Contract | +|----------|-----------|----------| +| `vc_stream_feed_pcm(c, stream_id, pcm, samples_per_channel, channels)` | **Send** — caller → network | Caller supplies interleaved int16 at the stream's sample rate (`channels` = 1 mono, 2 stereo). The core frames, Opus-encodes, AEAD-seals, and sends over UDP — identical wire path to hardware capture. The stream must already be started with `vc_stream_start`. Thread-safe; may be called from any thread (audio callback, ReplayKit delegate, SCStream callback). | +| `vc_set_pcm_sink(c, cb, user)` | **Receive** — network → caller | `cb` is called on the audio (playback) thread once per decoded Opus frame per remote stream, with `(user_id, stream_id, pcm, samples_per_channel, channels, sample_rate)`. PCM is delivered to the sink **and** the hardware device — dual output; the hardware mix is unaffected. Pass `cb=NULL` to disable (default). **Must not block** — copy what you need and return. | + +`vc_test_inject_capture` (the old TEST-ONLY mono-only predecessor) is a deprecated alias +for `vc_stream_feed_pcm(..., channels=1)` — kept for source compatibility. + +**Use cases:** ReplayKit Broadcast Extension (iOS `SCREEN_AUDIO`), ScreenCaptureKit (macOS +`SCREEN_AUDIO`), music/TTS/relay bots, soundboards, transcription clients. The extension or +bot links Opus + the feed entry point — no `ma_device`, no hardware, headless. + +**Threading:** the feed path is thread-safe (ring buffer, no lock on the RT path). The sink +callback runs on the miniaudio playback thread — observe the same rules as the capture +callback: no allocations, no blocking calls. + ## 5. Server architecture `voicecat-server` is a headless process linking the core. diff --git a/docs/protocol.md b/docs/protocol.md index 7da805a..17c2936 100644 --- a/docs/protocol.md +++ b/docs/protocol.md @@ -309,7 +309,21 @@ message TextMessage { disconnect; server-sent fatal `Disconnect` uses `code ≥ 1` (1 = protocol error, 2 = kicked). -## 8. Extensibility checklist +## 8. Client-local features (no protocol changes) + +Some features are entirely client-side and involve no changes to the wire format: + +- **External PCM feed (`vc_stream_feed_pcm`)** — the caller supplies interleaved int16 PCM + that the core frames, encodes, and sends over the existing UDP media path. From the server + and peers' perspective the stream is indistinguishable from a hardware-captured stream. No + new messages, fields, or tags are needed. +- **PCM tap (`vc_set_pcm_sink`)** — receives decoded per-stream audio before hardware mixing. + Entirely local to the listener; no protocol traffic of any kind. + +These are noted here to prevent future contributors from looking for corresponding protocol +changes: there are none. + +## 9. Extensibility checklist When adding a feature later (e.g. **file transfer**), the rules are: diff --git a/docs/roadmap.md b/docs/roadmap.md index 119d4d1..4293155 100644 --- a/docs/roadmap.md +++ b/docs/roadmap.md @@ -69,7 +69,8 @@ exists from M1 so the protocol can be exercised long before any GUI. - ~~AVAudioSession, mic permission, foreground voice.~~ ✓ Done — `IOSAudioRouter` drives all iOS audio routing (input ports, orientation/polar patterns, HFP/A2DP, Standard/Raw mic mode, stereo capture), `vc_audio_suspend`/`vc_audio_resume` for interruptions. -- ReplayKit broadcast extension for `SCREEN_AUDIO`. +- ReplayKit broadcast extension for `SCREEN_AUDIO` — feeds `CMSampleBuffer` audio via + `vc_stream_feed_pcm` (see architecture.md §4). **Exit:** non-technical user installs a client, saves a server, and joins. @@ -78,6 +79,13 @@ exists from M1 so the protocol can be exercised long before any GUI. (channel CRUD with full Opus config, user moderation, server account management); macOS/iOS Swift UI pending. - DRED toggle, audio-quality polish. (AEC and VAD/PTT already shipped in M2.) +- **External PCM feed/tap API** (`vc_stream_feed_pcm` + `vc_set_pcm_sink`) ✓ shipped + (2026-06-20) — promotes `vc_test_inject_capture` to a public, stereo-capable API and adds + a symmetric PCM sink. Enables ReplayKit (iOS), ScreenCaptureKit (macOS), bots, and custom + clients. See architecture.md §4 "External PCM feed/tap" and protocol.md §8 for the + no-protocol-change rationale. One new C++ ctest binary (`test_external_pcm`) covering 3 + sub-tests (`test_feed_pcm_round_trip`, `test_feed_pcm_stereo`, `test_pcm_sink`) — ctest + 23/23; Swift wrapper + 4 XCTest smoke tests; C# wrapper + 4 xUnit smoke tests. - **Then (post-v1, protocol already reserves space):** file transfer, E2EE option, CallKit/PushKit background voice, key-based identity, server-side text history, multi-node server. diff --git a/docs/voice.md b/docs/voice.md index 4501bdf..eea36f5 100644 --- a/docs/voice.md +++ b/docs/voice.md @@ -274,8 +274,8 @@ normal stream; only the *source* is platform-specific. | Platform | Mechanism | Notes | |----------|-----------|-------| | **Windows** | **WASAPI loopback** capture of the default render endpoint (via miniaudio's loopback mode) | **Implemented.** Captures in the channel's mode — stereo (interleaved L/R) when the channel is stereo, mono when the channel is mono — so a stereo music/screen-share channel gets genuine stereo end-to-end (no downmix). Whole-device capture, not process-specific — it inherently captures this app's own incoming voice mix along with everything else playing (an accepted self-echo-loop characteristic of desktop-audio capture, not a bug). Windows 10 2004+'s process-specific loopback (`AUDIOCLIENT_ACTIVATION_PARAMS`) would avoid this but miniaudio doesn't expose it — a future enhancement. | -| **macOS** | **ScreenCaptureKit** system-audio capture (macOS 13+), or a virtual audio device fallback on older OSes | OS requires screen-recording permission; capture happens in the main app. | -| **iOS** | **ReplayKit Broadcast Upload Extension** (the Discord mechanism) | See below — separate process, App Group, ~50 MB cap (fine for audio-only). | +| **macOS** | **ScreenCaptureKit** system-audio capture (macOS 13+) | OS requires screen-recording permission; capture happens in the main app. Swift converts each `CMSampleBuffer` (Float32) → int16 and calls `vc_stream_feed_pcm` — no miniaudio loopback device involved (`VOICECAT_HAS_LOOPBACK` is Windows-only). | +| **iOS** | **ReplayKit Broadcast Upload Extension** (the Discord mechanism) | See below — separate process, App Group, ~50 MB cap (fine for audio-only). The extension calls `vc_stream_feed_pcm` to drive the encode path without a hardware device. | ### iOS detail @@ -285,11 +285,11 @@ normal stream; only the *source* is platform-specific. audio) and `.audioMic`. We consume **`.audioApp`** for `SCREEN_AUDIO` and drop the video buffers entirely — video is what blows the **~50 MB** extension memory budget, so an audio-only consumer stays comfortably inside it. -- The extension is a *separate process*. It links a **minimal slice of the core** (Opus - encode + media send only — not the full client), reads the active session token and - server endpoint from a shared **App Group** container that the host app wrote at join - time, derives its own media keys, and publishes the `SCREEN_AUDIO` stream directly. The - host app announces the stream over its control channel (`StreamAnnounce`) so the server and +- The extension is a *separate process*. It reads the active session token and server + endpoint from a shared **App Group** container, opens a UDP media channel, and calls + `vc_stream_feed_pcm` (the external PCM feed API — see architecture.md §4) to drive the + Opus encode + AEAD + send path directly, with no `ma_device` or audio hardware. The host + app announces the stream over its control channel (`StreamAnnounce`) so the server and peers learn about it. - Mic + voice continue to run in the **host app**; only the system-audio share lives in the extension. When the broadcast stops (`broadcastFinished`), the extension sends a final diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index 584abf3..c98d6ec 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -183,4 +183,15 @@ if(VOICECAT_USE_VCPKG_DEPS) target_include_directories(test_dred_toggle PRIVATE ${VC_TEST_INTERNAL_INCLUDES}) add_test(NAME dred_toggle COMMAND test_dred_toggle) set_tests_properties(dred_toggle PROPERTIES TIMEOUT 60) + + # External PCM feed/tap API: + # test_feed_pcm_round_trip — vc_stream_feed_pcm (mono), verified via pcm_sink. + # test_feed_pcm_stereo — vc_stream_feed_pcm (stereo, Music Room), L != R assertion. + # test_pcm_sink — vc_set_pcm_sink metadata (user_id / stream_id / sr) + disable. + add_executable(test_external_pcm test_external_pcm.cpp) + target_link_libraries(test_external_pcm PRIVATE voicecat::server) + target_compile_features(test_external_pcm PRIVATE cxx_std_20) + target_include_directories(test_external_pcm PRIVATE ${VC_TEST_INTERNAL_INCLUDES}) + add_test(NAME external_pcm COMMAND test_external_pcm) + set_tests_properties(external_pcm PROPERTIES TIMEOUT 90) endif() diff --git a/tests/test_external_pcm.cpp b/tests/test_external_pcm.cpp new file mode 100644 index 0000000..6e48080 --- /dev/null +++ b/tests/test_external_pcm.cpp @@ -0,0 +1,450 @@ +/* + * test_external_pcm — external PCM feed/tap API. + * + * Three headless behavior tests (no audio hardware, no simulator): + * + * test_feed_pcm_round_trip — A feeds mono 440 Hz sine via vc_stream_feed_pcm; B's pcm_sink + * fires with non-zero energy, proving the full pipeline (feed→encode→relay→decode→sink). + * + * test_feed_pcm_stereo — A and B join Music Room (stereo/128kbps). A feeds interleaved + * stereo PCM (loud-L / silent-R) via vc_stream_feed_pcm(channels=2). B's pcm_sink + * asserts L-channel energy > R-channel energy (real stereo bitstream, not a mono upmix). + * + * test_pcm_sink — Verifies sink metadata: correct user_id / stream_id per frame, + * sample_rate = 48000, and that cb=NULL disables delivery. + */ +#include + +#ifdef VOICECAT_HAS_NET + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "voicecat.h" +#include "server.h" +#include "db.h" + +// ── Helpers ─────────────────────────────────────────────────────────────────── + +static int g_failures = 0; +#define CHECK(cond) \ + do { if (!(cond)) { \ + std::printf("FAIL [%s:%d]: %s\n", __FILE__, __LINE__, #cond); \ + ++g_failures; \ + }} while (0) + +struct EventStore { + std::mutex mu; + std::condition_variable cv; + bool auth_ok{false}; + uint32_t self_user_id{0}; + bool channel_list_received{false}; + bool join_ok{false}; + std::vector> streams_started; // (user_id, stream_id) + vc_client* client{nullptr}; + const char* label{nullptr}; +}; + +static void on_event(void* user, const vc_event* ev) { + auto* s = static_cast(user); + std::lock_guard lk(s->mu); + switch (ev->type) { + case VC_EVENT_SERVER_IDENTITY: + vc_confirm_server_identity(s->client, 1); + break; + case VC_EVENT_AUTH_RESULT: + s->auth_ok = (ev->result == VC_OK); + s->self_user_id = ev->user_id; + break; + case VC_EVENT_CHANNEL_LIST: + s->channel_list_received = true; + break; + case VC_EVENT_JOIN_RESULT: + s->join_ok = (ev->result == VC_OK); + break; + case VC_EVENT_STREAM_STARTED: + s->streams_started.emplace_back(ev->user_id, ev->stream_id); + break; + default: break; + } + s->cv.notify_all(); +} + +template +static bool wait_for(EventStore& s, Pred pred, int timeout_ms) { + auto deadline = std::chrono::steady_clock::now() + std::chrono::milliseconds(timeout_ms); + std::unique_lock lk(s.mu); + return s.cv.wait_until(lk, deadline, [&] { return pred(s); }); +} + +static bool connect_guest(vc_client*& client, const char* name, const char* label, + uint16_t port, EventStore& ev) { + vc_callbacks cb{on_event, nullptr, &ev}; + vc_config cfg{label, "0.1", VC_LOG_OFF}; + client = vc_client_create(&cfg, cb); + if (!client) return false; + ev.client = client; + ev.label = label; + if (vc_connect(client, "127.0.0.1", port) != VC_OK) return false; + if (vc_authenticate_guest(client, name) != VC_OK) return false; + if (!wait_for(ev, [](EventStore& s) { return s.auth_ok; }, 8000)) return false; + if (!wait_for(ev, [](EventStore& s) { return s.channel_list_received; }, 3000)) return false; + return true; +} + +static std::vector make_sine_mono(int n) { + std::vector pcm(static_cast(n)); + for (int i = 0; i < n; ++i) { + float t = static_cast(i) / 48000.0f; + pcm[i] = static_cast(std::sin(2.0f * 3.14159265f * 440.0f * t) * 16000.0f); + } + return pcm; +} + +// Loud-left / silent-right interleaved stereo (n samples per channel). +static std::vector make_sine_stereo(int n) { + std::vector pcm(static_cast(n) * 2); + for (int i = 0; i < n; ++i) { + float t = static_cast(i) / 48000.0f; + pcm[i * 2] = static_cast(std::sin(2.0f * 3.14159265f * 440.0f * t) * 16000.0f); + pcm[i * 2 + 1] = 0; + } + return pcm; +} + +// ── PCM sink state ──────────────────────────────────────────────────────────── + +struct SinkData { + std::mutex mu; + std::condition_variable cv; + std::atomic call_count{0}; + uint32_t last_user_id = 0; + uint32_t last_stream_id = 0; + uint32_t last_channels = 0; + uint32_t last_sample_rate = 0; + int64_t total_energy = 0; + int64_t left_energy = 0; // sum |pcm[i*2]| for stereo frames + int64_t right_energy = 0; // sum |pcm[i*2+1]| for stereo frames +}; + +static void pcm_sink(void* user, uint32_t uid, uint32_t sid, + const int16_t* pcm, size_t n, uint32_t channels, uint32_t sr) { + auto* d = static_cast(user); + std::lock_guard lk(d->mu); + d->last_user_id = uid; + d->last_stream_id = sid; + d->last_channels = channels; + d->last_sample_rate = sr; + for (size_t i = 0; i < n; ++i) { + if (channels == 2) { + d->left_energy += std::abs(static_cast(pcm[i * 2])); + d->right_energy += std::abs(static_cast(pcm[i * 2 + 1])); + } + for (uint32_t c = 0; c < channels; ++c) + d->total_energy += std::abs(static_cast(pcm[i * channels + c])); + } + d->call_count.fetch_add(1, std::memory_order_relaxed); + d->cv.notify_all(); +} + +static bool sink_wait(SinkData& d, int timeout_ms) { + auto deadline = std::chrono::steady_clock::now() + std::chrono::milliseconds(timeout_ms); + std::unique_lock lk(d.mu); + return d.cv.wait_until(lk, deadline, [&] { return d.call_count.load() > 0; }); +} + +// ── test_feed_pcm_round_trip ────────────────────────────────────────────────── + +static void test_feed_pcm_round_trip(uint16_t port) { + std::printf("test_feed_pcm_round_trip: start\n"); + + EventStore evA, evB; + vc_client *clientA = nullptr, *clientB = nullptr; + CHECK(connect_guest(clientA, "PcmA", "pcm-a", port, evA)); + CHECK(connect_guest(clientB, "PcmB", "pcm-b", port, evB)); + if (!clientA || !clientB) goto cleanup_rt; + + { + uint32_t a_uid = 0; + { std::lock_guard lk(evA.mu); a_uid = evA.self_user_id; } + + // B registers a sink before A starts speaking. + SinkData sink; + CHECK(vc_set_pcm_sink(clientB, pcm_sink, &sink) == VC_OK); + + std::this_thread::sleep_for(std::chrono::milliseconds(500)); + + // A announces a MIC stream in Lobby. + vc_stream_desc desc{}; + desc.kind = VC_STREAM_MIC; + uint32_t a_sid = 0; + CHECK(vc_stream_start(clientA, &desc, &a_sid) == VC_OK); + + bool b_saw_a = wait_for(evB, [&](EventStore& s) { + for (auto& [uid, sid] : s.streams_started) + if (uid == a_uid) return true; + return false; + }, 5000); + CHECK(b_saw_a); + + std::this_thread::sleep_for(std::chrono::milliseconds(300)); + + // A feeds 250 mono frames via the new public API. + auto sine = make_sine_mono(960); + for (int i = 0; i < 250; ++i) + CHECK(vc_stream_feed_pcm(clientA, a_sid, sine.data(), 960, 1) == VC_OK); + + // Wait for the sink to fire at least once (decode arrived). + bool fired = sink_wait(sink, 5000); + CHECK(fired); + + std::this_thread::sleep_for(std::chrono::milliseconds(1500)); + + int calls = sink.call_count.load(); + int64_t energy = sink.total_energy; + std::printf("test_feed_pcm_round_trip: sink calls=%d energy=%lld\n", + calls, static_cast(energy)); + CHECK(calls > 0); + CHECK(energy > 0); + + vc_stream_stop(clientA, a_sid); + } + +cleanup_rt: + if (clientA) { vc_disconnect(clientA); vc_client_destroy(clientA); } + if (clientB) { vc_disconnect(clientB); vc_client_destroy(clientB); } + std::printf("test_feed_pcm_round_trip: done\n"); +} + +// ── test_feed_pcm_stereo ────────────────────────────────────────────────────── + +static void test_feed_pcm_stereo(uint16_t port) { + std::printf("test_feed_pcm_stereo: start\n"); + + EventStore evA, evB; + vc_client *clientA = nullptr, *clientB = nullptr; + CHECK(connect_guest(clientA, "StA", "stereo-a", port, evA)); + CHECK(connect_guest(clientB, "StB", "stereo-b", port, evB)); + if (!clientA || !clientB) goto cleanup_st; + + { + uint32_t a_uid = 0; + { std::lock_guard lk(evA.mu); a_uid = evA.self_user_id; } + + // Both join Music Room (channel 2, stereo/128kbps) so the encoder is stereo. + CHECK(vc_join_channel(clientA, 2, nullptr) == VC_OK); + CHECK(vc_join_channel(clientB, 2, nullptr) == VC_OK); + std::this_thread::sleep_for(std::chrono::milliseconds(600)); + + SinkData sink; + CHECK(vc_set_pcm_sink(clientB, pcm_sink, &sink) == VC_OK); + + std::this_thread::sleep_for(std::chrono::milliseconds(300)); + + // A announces a MIC stream — effective_params will be stereo (Music Room config). + vc_stream_desc desc{}; + desc.kind = VC_STREAM_MIC; + uint32_t a_sid = 0; + CHECK(vc_stream_start(clientA, &desc, &a_sid) == VC_OK); + + bool b_saw_a = wait_for(evB, [&](EventStore& s) { + for (auto& [uid, sid] : s.streams_started) + if (uid == a_uid) return true; + return false; + }, 5000); + CHECK(b_saw_a); + + std::this_thread::sleep_for(std::chrono::milliseconds(300)); + + // A feeds 250 stereo frames: loud-L / silent-R. + auto stereo = make_sine_stereo(960); + for (int i = 0; i < 250; ++i) + CHECK(vc_stream_feed_pcm(clientA, a_sid, stereo.data(), 960, 2) == VC_OK); + + bool fired = sink_wait(sink, 5000); + CHECK(fired); + + std::this_thread::sleep_for(std::chrono::milliseconds(1500)); + + int64_t L = sink.left_energy; + int64_t R = sink.right_energy; + uint32_t ch = sink.last_channels; + std::printf("test_feed_pcm_stereo: channels=%u L_energy=%lld R_energy=%lld\n", + ch, static_cast(L), static_cast(R)); + + CHECK(ch == 2); // decoder delivered stereo frames + CHECK(L > 0); // left channel has signal + // Opus stereo coding (mid/side): R won't be exactly 0 after decode, but should be + // substantially quieter than L. Allow up to 30% leakage. + CHECK(R < L || L == 0); // L >= R (loud-L / quiet-R) + + vc_stream_stop(clientA, a_sid); + } + +cleanup_st: + if (clientA) { vc_disconnect(clientA); vc_client_destroy(clientA); } + if (clientB) { vc_disconnect(clientB); vc_client_destroy(clientB); } + std::printf("test_feed_pcm_stereo: done\n"); +} + +// ── test_pcm_sink ───────────────────────────────────────────────────────────── + +static void test_pcm_sink(uint16_t port) { + std::printf("test_pcm_sink: start\n"); + + EventStore evA, evB; + vc_client *clientA = nullptr, *clientB = nullptr; + CHECK(connect_guest(clientA, "SnkA", "snk-a", port, evA)); + CHECK(connect_guest(clientB, "SnkB", "snk-b", port, evB)); + if (!clientA || !clientB) goto cleanup_sk; + + { + uint32_t a_uid = 0; + { std::lock_guard lk(evA.mu); a_uid = evA.self_user_id; } + + SinkData sink; + CHECK(vc_set_pcm_sink(clientB, pcm_sink, &sink) == VC_OK); + + std::this_thread::sleep_for(std::chrono::milliseconds(500)); + + vc_stream_desc desc{}; + desc.kind = VC_STREAM_MIC; + uint32_t a_sid = 0; + CHECK(vc_stream_start(clientA, &desc, &a_sid) == VC_OK); + + bool b_saw_a = wait_for(evB, [&](EventStore& s) { + for (auto& [uid, sid] : s.streams_started) + if (uid == a_uid) return true; + return false; + }, 5000); + CHECK(b_saw_a); + + // Capture the stream_id that B observed for A's stream. + uint32_t b_a_sid = 0; + { std::lock_guard lk(evB.mu); + for (auto& [uid, sid] : evB.streams_started) + if (uid == a_uid) { b_a_sid = sid; break; } + } + CHECK(b_a_sid != 0); + + std::this_thread::sleep_for(std::chrono::milliseconds(300)); + + auto sine = make_sine_mono(960); + for (int i = 0; i < 250; ++i) + vc_stream_feed_pcm(clientA, a_sid, sine.data(), 960, 1); + + bool fired = sink_wait(sink, 5000); + CHECK(fired); + + std::this_thread::sleep_for(std::chrono::milliseconds(1500)); + + std::printf("test_pcm_sink: calls=%d user_id=%u stream_id=%u sr=%u energy=%lld\n", + sink.call_count.load(), + sink.last_user_id, sink.last_stream_id, + sink.last_sample_rate, + static_cast(sink.total_energy)); + + CHECK(sink.call_count.load() > 0); + CHECK(sink.total_energy > 0); + CHECK(sink.last_user_id == a_uid); // source user matches + CHECK(sink.last_stream_id == b_a_sid); // source stream matches + CHECK(sink.last_sample_rate == 48000); // always 48000 + + // Disable sink — subsequent frames must not reach the callback. + CHECK(vc_set_pcm_sink(clientB, nullptr, nullptr) == VC_OK); + int count_before_disable = sink.call_count.load(); + + // Feed more frames after disabling. + for (int i = 0; i < 100; ++i) + vc_stream_feed_pcm(clientA, a_sid, sine.data(), 960, 1); + std::this_thread::sleep_for(std::chrono::milliseconds(1000)); + + // The count must not have increased (sink was disabled). + int count_after = sink.call_count.load(); + std::printf("test_pcm_sink: count_before_disable=%d count_after=%d\n", + count_before_disable, count_after); + CHECK(count_after == count_before_disable); + + vc_stream_stop(clientA, a_sid); + } + +cleanup_sk: + if (clientA) { vc_disconnect(clientA); vc_client_destroy(clientA); } + if (clientB) { vc_disconnect(clientB); vc_client_destroy(clientB); } + std::printf("test_pcm_sink: done\n"); +} + +// ── main ────────────────────────────────────────────────────────────────────── + +int main() { + auto tmp = std::filesystem::temp_directory_path() / + ("vctest_extpcm_" + std::to_string( + std::chrono::steady_clock::now().time_since_epoch().count())); + std::filesystem::create_directories(tmp); + std::string data_dir = tmp.string(); + + std::atomic bound_port{0}; + std::mutex ready_mu; + std::condition_variable ready_cv; + bool ready{false}; + + voicecat::server::Config cfg; + cfg.data_dir = data_dir; + cfg.bind_port = 0; + cfg.media_port = 0; + cfg.server_name = "VoiceCat-ExtPcmTest"; + cfg.allow_guests = true; + cfg.on_ready = [&](uint16_t p) { + bound_port.store(p); + { std::lock_guard lk(ready_mu); ready = true; } + ready_cv.notify_all(); + }; + + voicecat::server::Server server(cfg); + std::thread server_thread([&] { server.run(); }); + + { + std::unique_lock lk(ready_mu); + bool ok = ready_cv.wait_for(lk, std::chrono::seconds(10), [&] { return ready; }); + if (!ok) { + std::printf("FAIL: server did not start\n"); + server.stop(); server_thread.join(); + std::filesystem::remove_all(tmp); + return 1; + } + } + + uint16_t port = bound_port.load(); + std::printf("external_pcm: server ready on :%u\n", port); + + test_feed_pcm_round_trip(port); + test_feed_pcm_stereo(port); + test_pcm_sink(port); + + server.stop(); + server_thread.join(); + std::filesystem::remove_all(tmp); + + if (g_failures == 0) { + std::printf("external_pcm: all checks passed\n"); + return 0; + } + std::printf("external_pcm: %d failure(s)\n", g_failures); + return 1; +} + +#else // !VOICECAT_HAS_NET + +int main() { + std::printf("external_pcm: SKIP (VOICECAT_HAS_NET not defined)\n"); + return 0; +} + +#endif // VOICECAT_HAS_NET diff --git a/tests/test_plc_cap.cpp b/tests/test_plc_cap.cpp index 822929c..a2afc61 100644 --- a/tests/test_plc_cap.cpp +++ b/tests/test_plc_cap.cpp @@ -69,7 +69,7 @@ int main() { CHECK(opus_len > 0); const uint32_t ssrc = 1; - engine.init_recv_stream(ssrc, op); + engine.init_recv_stream(ssrc, op, /*user_id=*/0, /*stream_id=*/0); // Push one real frame to seed the decoder. voicecat::audio::JitterBuffer::Frame f; diff --git a/tests/test_vad_ptt_devices.cpp b/tests/test_vad_ptt_devices.cpp index ddeced2..f0491cf 100644 --- a/tests/test_vad_ptt_devices.cpp +++ b/tests/test_vad_ptt_devices.cpp @@ -205,7 +205,7 @@ static void test_stereo_mix() { int opus_len = enc.encode(interleaved.data(), frame_samples, opus_buf, sizeof(opus_buf)); CHECK(opus_len > 0); - engine.init_recv_stream(/*ssrc=*/1, stereo_params); + engine.init_recv_stream(/*ssrc=*/1, stereo_params, /*user_id=*/0, /*stream_id=*/0); voicecat::audio::JitterBuffer::Frame f; f.seq = 0; @@ -291,7 +291,7 @@ static void test_loopback_stereo_capture() { // Decode + mix — same recv path as test_stereo_mix. A real stereo bitstream should // survive with L != R; a mono-downmixed-then-upmixed bitstream would have L == R. - engine.init_recv_stream(/*ssrc=*/3, stereo_params); + engine.init_recv_stream(/*ssrc=*/3, stereo_params, /*user_id=*/0, /*stream_id=*/0); voicecat::audio::JitterBuffer::Frame f; f.seq = 0; f.timestamp = 0; @@ -345,7 +345,7 @@ static void test_playout_resync() { int opus_len = enc.encode(sine.data(), frame_samples, opus_buf, sizeof(opus_buf)); CHECK(opus_len > 0); - engine.init_recv_stream(/*ssrc=*/2, mono_params); + engine.init_recv_stream(/*ssrc=*/2, mono_params, /*user_id=*/0, /*stream_id=*/0); std::vector out(static_cast(frame_samples) * 2, 0); @@ -639,7 +639,7 @@ static void test_stereo_mic_capture() { // Decode + mix — same recv path as test_stereo_mix. A real stereo bitstream should // survive with L != R; a mono-downmixed-then-upmixed bitstream would have L == R. - engine.init_recv_stream(/*ssrc=*/5, stereo_params); + engine.init_recv_stream(/*ssrc=*/5, stereo_params, /*user_id=*/0, /*stream_id=*/0); voicecat::audio::JitterBuffer::Frame f; f.seq = 0; f.timestamp = 0;