feat: external PCM feed/tap API (vc_stream_feed_pcm + vc_set_pcm_sink)
Promotes vc_test_inject_capture (mono-only, TEST-ONLY) to a public, stereo-capable production API and adds a symmetric PCM tap on the receive side. Enables ReplayKit (iOS), ScreenCaptureKit (macOS), bots, soundboards, and custom clients — all without a hardware audio device. Core C++: - voicecat.h: new vc_stream_feed_pcm, vc_pcm_sink_cb typedef, vc_set_pcm_sink; vc_test_inject_capture kept as deprecated alias - audio_engine: stereo-aware inject_capture (channels param + ring reset on channel-count change); atomic pcm_sink_ fired per decoded frame in on_playback; RemoteStream carries user_id/stream_id for RT-safe sink metadata; init_recv_stream takes user_id+stream_id - client.cpp: stream_feed_pcm / set_pcm_sink implementations; sync_remote_streams passes user_id/stream_id to init_recv_stream - voicecat.cpp: trampolines + channels=1/2 validation Tests: test_external_pcm (headless, 3 sub-tests: mono round-trip, stereo feed L≠R, sink metadata+disable). ctest 23/23. Swift: feedPcm / setPcmSink in VoiceCatClient.swift + 4 XCTest smoke tests (ExternalPcmTests.swift). C#: StreamFeedPcm / SetPcmSink in VoiceCatClient.cs + NativeMethods.cs (vc_stream_feed_pcm unsafe P/Invoke, VcPcmSinkCallback delegate, vc_set_pcm_sink via nint) + 4 xUnit smoke tests (ExternalPcmTests.cs). Docs: architecture.md §4 new subsection, voice.md §9 updated (macOS/iOS now reference vc_stream_feed_pcm), protocol.md §8 explicit no-protocol-change note, roadmap.md M5 entry. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
@@ -155,6 +155,26 @@ Design notes:
|
||||
`PostMessage` with no meaningful latency cost. `VoiceCatClientHandle : SafeHandle` wraps
|
||||
the `vc_client*` and guarantees `vc_client_destroy` runs on GC/Dispose.
|
||||
|
||||
### External PCM feed/tap
|
||||
|
||||
Two API functions let callers bypass miniaudio entirely for a stream:
|
||||
|
||||
| Function | Direction | Contract |
|
||||
|----------|-----------|----------|
|
||||
| `vc_stream_feed_pcm(c, stream_id, pcm, samples_per_channel, channels)` | **Send** — caller → network | Caller supplies interleaved int16 at the stream's sample rate (`channels` = 1 mono, 2 stereo). The core frames, Opus-encodes, AEAD-seals, and sends over UDP — identical wire path to hardware capture. The stream must already be started with `vc_stream_start`. Thread-safe; may be called from any thread (audio callback, ReplayKit delegate, SCStream callback). |
|
||||
| `vc_set_pcm_sink(c, cb, user)` | **Receive** — network → caller | `cb` is called on the audio (playback) thread once per decoded Opus frame per remote stream, with `(user_id, stream_id, pcm, samples_per_channel, channels, sample_rate)`. PCM is delivered to the sink **and** the hardware device — dual output; the hardware mix is unaffected. Pass `cb=NULL` to disable (default). **Must not block** — copy what you need and return. |
|
||||
|
||||
`vc_test_inject_capture` (the old TEST-ONLY mono-only predecessor) is a deprecated alias
|
||||
for `vc_stream_feed_pcm(..., channels=1)` — kept for source compatibility.
|
||||
|
||||
**Use cases:** ReplayKit Broadcast Extension (iOS `SCREEN_AUDIO`), ScreenCaptureKit (macOS
|
||||
`SCREEN_AUDIO`), music/TTS/relay bots, soundboards, transcription clients. The extension or
|
||||
bot links Opus + the feed entry point — no `ma_device`, no hardware, headless.
|
||||
|
||||
**Threading:** the feed path is thread-safe (ring buffer, no lock on the RT path). The sink
|
||||
callback runs on the miniaudio playback thread — observe the same rules as the capture
|
||||
callback: no allocations, no blocking calls.
|
||||
|
||||
## 5. Server architecture
|
||||
|
||||
`voicecat-server` is a headless process linking the core.
|
||||
|
||||
Reference in New Issue
Block a user