feat: external PCM feed/tap API (vc_stream_feed_pcm + vc_set_pcm_sink)
Promotes vc_test_inject_capture (mono-only, TEST-ONLY) to a public, stereo-capable production API and adds a symmetric PCM tap on the receive side. Enables ReplayKit (iOS), ScreenCaptureKit (macOS), bots, soundboards, and custom clients — all without a hardware audio device. Core C++: - voicecat.h: new vc_stream_feed_pcm, vc_pcm_sink_cb typedef, vc_set_pcm_sink; vc_test_inject_capture kept as deprecated alias - audio_engine: stereo-aware inject_capture (channels param + ring reset on channel-count change); atomic pcm_sink_ fired per decoded frame in on_playback; RemoteStream carries user_id/stream_id for RT-safe sink metadata; init_recv_stream takes user_id+stream_id - client.cpp: stream_feed_pcm / set_pcm_sink implementations; sync_remote_streams passes user_id/stream_id to init_recv_stream - voicecat.cpp: trampolines + channels=1/2 validation Tests: test_external_pcm (headless, 3 sub-tests: mono round-trip, stereo feed L≠R, sink metadata+disable). ctest 23/23. Swift: feedPcm / setPcmSink in VoiceCatClient.swift + 4 XCTest smoke tests (ExternalPcmTests.swift). C#: StreamFeedPcm / SetPcmSink in VoiceCatClient.cs + NativeMethods.cs (vc_stream_feed_pcm unsafe P/Invoke, VcPcmSinkCallback delegate, vc_set_pcm_sink via nint) + 4 xUnit smoke tests (ExternalPcmTests.cs). Docs: architecture.md §4 new subsection, voice.md §9 updated (macOS/iOS now reference vc_stream_feed_pcm), protocol.md §8 explicit no-protocol-change note, roadmap.md M5 entry. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
32
PROGRESS.md
32
PROGRESS.md
@@ -39,12 +39,12 @@ up instantly. Newest status at the top.
|
||||
`allowBluetoothA2DP` and the output route must remain the headphones/A2DP device — NOT flip
|
||||
to `…Record`. If confirmed, delete the `logSessionState` calls + method and the prior
|
||||
band-aid comments in `IOSAudioRouter`/`audio_engine.cpp` can be trimmed.
|
||||
- **Verified on Windows:** `cmake --build --preset dev` clean, `ctest --preset dev` 22/22
|
||||
(21/21 prior + new `test_dred_toggle`).
|
||||
iOS build & on-device run still to be done by the user on the Mac.
|
||||
- **Verified on Windows:** `cmake --build --preset dev` clean, `ctest --preset dev` 23/23
|
||||
(22/22 prior + `test_external_pcm` new binary). iOS build & on-device run still to be done by the user on the Mac.
|
||||
|
||||
- **Planned (not started):** **External PCM feed/tap API (`vc_stream_feed_pcm` +
|
||||
`vc_set_pcm_sink`)** (2026-06-19, plan written on Windows; implement on Mac). A public,
|
||||
- **Done (2026-06-20):** **External PCM feed/tap API (`vc_stream_feed_pcm` +
|
||||
`vc_set_pcm_sink`)** — see detail in M5 section below. `ctest --preset dev` 23/23 (was 22/22 + 1 new test binary with 3 sub-tests).
|
||||
Next: iOS ReplayKit and macOS ScreenCaptureKit consumers of this API. A public,
|
||||
documented API for driving audio streams with externally-provided PCM instead of (or in
|
||||
addition to) miniaudio's hardware device. Motivated by four concrete use cases — all in our
|
||||
roadmap — that the current "miniaudio owns the device" model can't serve:
|
||||
@@ -208,7 +208,17 @@ up instantly. Newest status at the top.
|
||||
|
||||
## Recent completed work
|
||||
|
||||
All items below are `[x]` done; `ctest --preset dev` 21/21 on Windows after each.
|
||||
All items below are `[x]` done; `ctest --preset dev` 23/23 on Windows after all.
|
||||
|
||||
- **External PCM feed/tap API** (2026-06-20): `vc_stream_feed_pcm` + `vc_set_pcm_sink` shipped.
|
||||
Promotes `vc_test_inject_capture` (mono-only, TEST-ONLY) to a public, stereo-capable API.
|
||||
Adds symmetric PCM sink on the playback thread. Swift wrapper (`feedPcm`/`setPcmSink` in
|
||||
`VoiceCatClient.swift`, 4 XCTest smoke tests). C# wrapper (`StreamFeedPcm`/`SetPcmSink` in
|
||||
`VoiceCatClient.cs` + `NativeMethods.cs`, 4 xUnit smoke tests in `ExternalPcmTests.cs`).
|
||||
Three new headless C++ ctests. Docs: architecture.md §4 new subsection, voice.md §9 updated,
|
||||
protocol.md §8 explicit no-protocol-change note, roadmap.md M5 entry. Files: `voicecat.h`,
|
||||
`voicecat.cpp`, `client.{h,cpp}`, `audio_engine.{h,cpp}`, `tests/test_external_pcm.cpp`,
|
||||
`tests/CMakeLists.txt`, Swift + C# wrappers.
|
||||
|
||||
- **iOS A2DP + stereo root cause fix** (2026-06-20): miniaudio's NULL-context `ma_device_init`
|
||||
was calling `AVAudioSession setCategory(Record)` on every device open, wiping the session
|
||||
@@ -412,8 +422,14 @@ iOS 18.0 deployment target. App Group `group.cat.voice.VoiceCat` for Keychain sh
|
||||
on macOS). Implement via `vc_stream_feed_pcm` + `SCStream` once the feed API ships.
|
||||
- [ ] **iOS ReplayKit Broadcast Extension** (`VoiceCatBroadcast`) — separate Xcode target,
|
||||
App Group credential sharing, `SampleHandler.swift`. Implement via `vc_stream_feed_pcm`.
|
||||
- [ ] **External PCM feed/tap API** (`vc_stream_feed_pcm` + `vc_set_pcm_sink`) — see full
|
||||
plan in "Where we left off" above.
|
||||
- [x] **External PCM feed/tap API** (`vc_stream_feed_pcm` + `vc_set_pcm_sink`) — done
|
||||
2026-06-20. Promotes `vc_test_inject_capture` (mono-only, TEST-ONLY) to a public API with
|
||||
stereo support. Adds a symmetric PCM sink fired on the playback thread per decoded remote
|
||||
stream. Full wrappers for Swift (`feedPcm`/`setPcmSink`) and C# (`StreamFeedPcm`/
|
||||
`SetPcmSink`). Three new C++ ctests (`test_feed_pcm_round_trip`, `test_feed_pcm_stereo`,
|
||||
`test_pcm_sink`), 4 Swift XCTest smoke tests, 4 C# xUnit smoke tests. Docs updated
|
||||
(architecture.md §4 new subsection, voice.md §9 updated, protocol.md §8 explicit
|
||||
no-protocol-change note, roadmap.md M5 entry). `ctest --preset dev` 23/23.
|
||||
|
||||
---
|
||||
|
||||
|
||||
Reference in New Issue
Block a user