Talon/voice-cat
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

feat(macos): per-app audio selection for screen sharing

Browse Source 
Let users choose what the SCREEN_AUDIO stream captures before sharing:
share everything, only selected apps, or all except selected apps, plus a
first-class "Exclude screen reader (VoiceOver) audio" toggle.

ScreenCaptureKit filters audio per application, so ScreenAudioCapture now
takes a ScreenAudioSelection and builds the matching SCContentFilter
(including:/excludingApplications:). New ScreenSharePickerSheet lists
running apps from SCShareableContent. iOS left untouched -- ReplayKit only
delivers the mixed system stream, so per-app filtering is impossible there.
This commit is contained in:
Talon
2026-06-21 13:35:01 +02:00
parent 6b7f06a282
commit c1e6f4f7ff
6 changed files with 389 additions and 16 deletions
Download Patch File Download Diff File Expand all files Collapse all files

20
docs/voice.md
View File

@@ -274,8 +274,24 @@ normal stream; only the *source* is platform-specific.
| Platform | Mechanism | Notes |
|----------|-----------|-------|
| **Windows** | **WASAPI loopback** capture of the default render endpoint (via miniaudio's loopback mode) | **Implemented.** Captures in the channel's mode — stereo (interleaved L/R) when the channel is stereo, mono when the channel is mono — so a stereo music/screen-share channel gets genuine stereo end-to-end (no downmix). Whole-device capture, not process-specific — it inherently captures this app's own incoming voice mix along with everything else playing (an accepted self-echo-loop characteristic of desktop-audio capture, not a bug). Windows 10 2004+'s process-specific loopback (`AUDIOCLIENT_ACTIVATION_PARAMS`) would avoid this but miniaudio doesn't expose it — a future enhancement. |
| **macOS** | **ScreenCaptureKit** system-audio capture (macOS 13+) | **Implemented** (`clients/apple/macOS/VoiceCatMac/Audio/ScreenAudioCapture.swift`). OS requires screen-recording permission; capture happens in the main app. An `SCStream` with `capturesAudio` + `excludesCurrentProcessAudio` delivers audio `CMSampleBuffer`s; Swift converts Float32 → int16 (in the channel's mono/stereo mode) and calls `vc_stream_feed_pcm` — no miniaudio loopback device involved (`VOICECAT_HAS_LOOPBACK` is Windows-only). |
| **iOS** | **ReplayKit Broadcast Upload Extension** (the Discord mechanism) | **Implemented.** See below — separate process, App Group, ~50 MB cap (fine for audio-only). |
| **macOS** | **ScreenCaptureKit** system-audio capture (macOS 13+) | **Implemented** (`clients/apple/macOS/VoiceCatMac/Audio/ScreenAudioCapture.swift`). OS requires screen-recording permission; capture happens in the main app. An `SCStream` with `capturesAudio` + `excludesCurrentProcessAudio` delivers audio `CMSampleBuffer`s; Swift converts Float32 → int16 (in the channel's mono/stereo mode) and calls `vc_stream_feed_pcm` — no miniaudio loopback device involved (`VOICECAT_HAS_LOOPBACK` is Windows-only). **Supports per-app audio selection** — see below. |
| **iOS** | **ReplayKit Broadcast Upload Extension** (the Discord mechanism) | **Implemented.** See below — separate process, App Group, ~50 MB cap (fine for audio-only). ReplayKit only ever delivers the *mixed* system stream as `.audioApp`, so **per-app filtering / VoiceOver exclusion is not possible on iOS** (it has no per-app granularity, unlike ScreenCaptureKit). |
### macOS detail — per-app audio selection
ScreenCaptureKit filters audio at the **application** level, so before sharing starts the user
picks a scope in `ScreenSharePickerSheet` (`clients/apple/macOS/VoiceCatMac/Sheets/`):
- **Everything** — whole display, the original behaviour (`SCContentFilter(display:excludingWindows:)`).
- **Only selected apps** — capture just the ticked apps (`init(display:including:exceptingWindows:)`).
- **All except selected apps** — capture everything but the ticked apps
(`init(display:excludingApplications:exceptingWindows:)`).
A dedicated **"Exclude screen reader (VoiceOver) audio"** toggle merges the screen-reader
process(es) into the exclude set (`ScreenAudioCapture.screenReaderBundleIDs` — VoiceOver plus
the speech-synthesis daemon that actually renders the spoken audio). The chosen
`ScreenAudioSelection` is passed into `ScreenAudioCapture`, which builds the matching
`SCContentFilter`. iOS/ReplayKit has no equivalent control (see the table note above).
### iOS detail