feat: external PCM feed/tap API (vc_stream_feed_pcm + vc_set_pcm_sink)
Promotes vc_test_inject_capture (mono-only, TEST-ONLY) to a public, stereo-capable production API and adds a symmetric PCM tap on the receive side. Enables ReplayKit (iOS), ScreenCaptureKit (macOS), bots, soundboards, and custom clients — all without a hardware audio device. Core C++: - voicecat.h: new vc_stream_feed_pcm, vc_pcm_sink_cb typedef, vc_set_pcm_sink; vc_test_inject_capture kept as deprecated alias - audio_engine: stereo-aware inject_capture (channels param + ring reset on channel-count change); atomic pcm_sink_ fired per decoded frame in on_playback; RemoteStream carries user_id/stream_id for RT-safe sink metadata; init_recv_stream takes user_id+stream_id - client.cpp: stream_feed_pcm / set_pcm_sink implementations; sync_remote_streams passes user_id/stream_id to init_recv_stream - voicecat.cpp: trampolines + channels=1/2 validation Tests: test_external_pcm (headless, 3 sub-tests: mono round-trip, stereo feed L≠R, sink metadata+disable). ctest 23/23. Swift: feedPcm / setPcmSink in VoiceCatClient.swift + 4 XCTest smoke tests (ExternalPcmTests.swift). C#: StreamFeedPcm / SetPcmSink in VoiceCatClient.cs + NativeMethods.cs (vc_stream_feed_pcm unsafe P/Invoke, VcPcmSinkCallback delegate, vc_set_pcm_sink via nint) + 4 xUnit smoke tests (ExternalPcmTests.cs). Docs: architecture.md §4 new subsection, voice.md §9 updated (macOS/iOS now reference vc_stream_feed_pcm), protocol.md §8 explicit no-protocol-change note, roadmap.md M5 entry. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -413,6 +413,45 @@ VC_API vc_result vc_test_inject_capture(vc_client* c, uint32_t stream_id, const
|
||||
* VC_ERR_INVALID_ARG if stream_id is unknown or channels is not 1 or 2. */
|
||||
VC_API vc_result vc_set_capture_channels(vc_client* c, uint32_t stream_id, uint32_t channels);
|
||||
|
||||
/* ── External PCM feed/tap ─────────────────────────────────────────────────── */
|
||||
|
||||
/* External PCM feed — production-grade API for driving a local stream's encode pipeline
|
||||
* with caller-supplied PCM instead of (or in addition to) a hardware capture device. The
|
||||
* stream must already be started (vc_stream_start). The core frames, encodes (Opus), seals
|
||||
* (AEAD), and sends (UDP) the provided samples exactly as it would mic/loopback audio.
|
||||
*
|
||||
* samples_per_channel : samples per channel (e.g. 960 for 20 ms @ 48 kHz).
|
||||
* channels : 1 (mono) or 2 (stereo interleaved L/R). VC_ERR_INVALID_ARG otherwise.
|
||||
*
|
||||
* Use cases: ReplayKit Broadcast Extension (iOS), ScreenCaptureKit (macOS), bots (TTS /
|
||||
* music / relay), soundboards, DAW integration. Works for any stream kind (MIC /
|
||||
* SCREEN_AUDIO / AUX_DEVICE). Thread-safe; may be called from any thread.
|
||||
*
|
||||
* Replaces vc_test_inject_capture (deprecated alias, see below). */
|
||||
VC_API vc_result vc_stream_feed_pcm(vc_client* c, uint32_t stream_id,
|
||||
const int16_t* pcm, size_t samples_per_channel,
|
||||
uint32_t channels);
|
||||
|
||||
/* External PCM tap — receive decoded remote audio as int16 PCM per stream, before it is
|
||||
* summed into the hardware mix. The callback fires on the audio playback thread once per
|
||||
* decoded Opus frame (typically every 20 ms) for each active remote stream:
|
||||
*
|
||||
* cb(user, user_id, stream_id, pcm, samples_per_channel, channels, sample_rate)
|
||||
*
|
||||
* user_id / stream_id : identify the sender (same values as VC_EVENT_STREAM_STARTED).
|
||||
* pcm : decoded int16 PCM, interleaved when channels == 2.
|
||||
* samples_per_channel : samples per channel for this frame (typically 960 @ 48 kHz).
|
||||
* channels : 1 or 2, matching the sender's stream configuration.
|
||||
* sample_rate : always 48000 in the current implementation.
|
||||
*
|
||||
* Pass cb = NULL to disable (default: disabled; hardware playback only).
|
||||
* The callback MUST NOT block, lock, or allocate — copy what you need and return.
|
||||
* PCM is still delivered to the hardware playback device regardless (dual output). */
|
||||
typedef void (*vc_pcm_sink_cb)(void* user, uint32_t user_id, uint32_t stream_id,
|
||||
const int16_t* pcm, size_t samples_per_channel,
|
||||
uint32_t channels, uint32_t sample_rate);
|
||||
VC_API vc_result vc_set_pcm_sink(vc_client* c, vc_pcm_sink_cb cb, void* user);
|
||||
|
||||
/* ── Text ─────────────────────────────────────────────────────────────────── */
|
||||
VC_API vc_result vc_send_text(vc_client* c, vc_text_scope scope, uint32_t target_id,
|
||||
const char* utf8);
|
||||
|
||||
Reference in New Issue
Block a user